Cited By
View all- khan Miftikhar SIqbal MSherin S(2018)Empirical studies omit reporting necessary detailsComputer Standards & Interfaces10.1016/j.csi.2017.08.00255:C(156-170)Online publication date: 1-Jan-2018
Scenario-based testing using symbolic animation of B models
Pages 407 - 434
Abstract
This article presents a model-based test generation technique, from user-defined scenarios, for behavioral models expressed as B machines. Scenarios are expressed using a customized formalism, based on regular expressions, that makes it possible to describe sequences of operation calls possibly reaching specific states of the system. A symbolic animation engine, simulating the execution of a model using constraint logic programming, is then exploited to play the unfolded scenarios on the model and to instantiate the test cases, providing the expected results used to establish the conformance verdict. This approach is tool supported by a research prototype and has been successfully applied in an industrial context of a smart card applet. This tool is extended by a scenario generator, which automatically generates testing strategies for exercising user-defined properties, written using specific patterns. Copyright © 2012 John Wiley & Sons, Ltd.
References
[1]
Beizer B. Black-box Testing: Techniques for Functional Testing of Software and Systems. John Wiley & Sons: New York, USA, 1995.
[2]
Utting M, Legeard B. Practical Model-based Testing – A Tools Approach. Elsevier Science, 2006. 550 pages.
[3]
Börger E, Stärk RF. Abstract State Machines. A method for High-level System Design and Analysis. Springer, 2003.
[4]
Abrial JR. The B-Book. Cambridge University Press, 1996.
[5]
Jaffuel E, Legeard B. LEIRIOS test generator: automated test generation from B models. B'2007, the 7th int. B Conference - Industrial Tool Session, volume 4355 of LNCS, Besancon, France, January 2007; 277–280. Springer.
[6]
Satpathy M, Leuschel M, Butler M. ProTest: an automatic test environment for B specifications. Electronic Notes in Theroretical Computer Science 2005 January; 111: 113–136.
[7]
Leuschel M, Butler M. 2003. ProB: a model checker for B. In FME 2003: Formal methods, Vol. 2805, Araki K, Gnesi S, Mandrioli D (eds), LNCS. Springer.
[8]
Colin S, Legeard B, Peureux F. Preamble computation in automated test case generation using constraint logic programming. The Journal of Software Testing, Verification and Reliability 2004; 14(3): 213–235.
[9]
Jöbstl E, Weiglhofer M, Aichernig BK, Wotawa F. When BDDS fail: conformance testing with symbolic execution and SMT solving. Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation, ICST '10, Washington, DC, USA, 2010; 479–488. IEEE Computer Society.
[10]
Julliand J, Masson P-A, Tissot R. Generating security tests in addition to functional tests. Ast'08, 3rd Int. Workshop on Automation of Software Test, Leipzig, Germany, May 2008; 41–44. ACM Press.
[11]
Dadeau F, Tissot R. jSynoPSys – a scenario-based testing tool based on the symbolic animation of B machines. In Mbt'09 proceedings, Vol. 253-2, Finkbeiner B, Gurevich Y, Petrenko AK (eds), ENTCS, Electronic Notes in Theoretical Computer Science: York, United Kingdom, October 2009; 117–132.
[12]
Rumbaugh J, Jacobson I, Booch G. The Unified Modeling Language Reference Manual, addison-wesley, 1999.
[13]
Leavens GT, Baker AL, Ruby C. 1999. JML: a notation for detailed design. In Behavioral specifications of businesses and systems, Kilov H, Rumpe B, Simmonds I (eds). Kluwer Academic Publishers: Boston, 175–188.
[14]
Behm P, Benoit P, Faivre A, Meynadier J-M. Météor: a successful application of B in a large project, Fm'99 -Formal Methods -volume 1, number 1708 in LNCS, September 1999; 348–387. Springer.
[15]
Lanet J-L, Requet A. Formal proof of smart card applets correctness, Cardis '98: Proceedings of the International Conference on Smart Card Research and Applications, London, UK, 2000; 85–97. Springer-Verlag.
[16]
Marlet R, Metayer DL. 2001. Security properties and Java Card specificities to be studied in the SecSafe project, Technical Report, SECSAFETL-006, Trusted Logics.
[17]
Bouquet F, Legeard B. Reification of executable test scripts in formal specification-based test generation: the Java Card transaction mechanism case study. Proc. of Fme'03, Formal Method Europe, volume 2805 of LNCS, Pisa, Italy, September 2003; 778–795.
[18]
Smart card standard: Part 4: interindustry commands for interchange. Technical Report, ISO/IEC, 1995.
[19]
Dick J, Faivre A. Automating the generation and sequencing of test cases from model-based specifications. In FME '93: First International Symposium of Formal Methods Europe, Vol. 670, Woodcock J, Gorm Larsen P (eds), LNCS, 1993 April; 268–284.
[20]
Bouquet F, Legeard B, Peureux F. CLPS-B: a constraint solver to animate a B specification. International Journal on Software Tools for Technology Transfer, STTT August 2004 6(2): 143–157.
[21]
Ambert F, Bouquet F, Chemin S, Guenaud S, Legeard B, Peureux F, Vacelet N, Utting M. BZ-TT: a tool-set for test generation from Z and B using constraint logic programming. Proc. of Formal Approaches to Testing of Software, FATES 2002 (workshop of concur'02), Brnö, République Tchèque, August 2002 105–120. INRIA report.
[22]
Jaffar J, Maher MJ. Constraint logic programming: a survey. Journal of Logic Programming 1994May/July; 19/20: 503–582.
[23]
Tsang E. Foundations of Constraint Satisfaction. Academic Press, 1993.
[24]
Bernard E, Legeard B, Luck X, Peureux F. Generation of test sequences from formal specifications: GSM 11-11 standard case study. International Journal of Software Practice and Experience 2004; 34(10): 915–948.
[25]
Ledru Y, du Bousquet L, Maury O, Bontron P. 2004. Filtering TOBIAS combinatorial test suites. In Fundamental Approaches to Software Engineering, 7th Int. Conf., FASE 2004, Vol. 2984, Wermelinger M, Margaria T (eds), LNCS, Springer, Barcelona, Spain.
[26]
Dwyer MB, Avrunin GS, Corbett JC. Patterns in property specifications for finite-state verification. Icse'99: Proceedings of the 21st international conference on software engineering, Los Alamitos, CA, USA, 1999; 411–420. IEEE Computer Society Press.
[27]
Bert D, Potet M-L, Stouls N. 2005. GeneSyst: a tool to reason about behavioral aspects of B event specifications. Application to security properties. In ZB 2005: Formal Specification and Development in Z and B, 4th International Conference of B and Z Users, Vol. 3455 of Lecture Notes in Computer Science, Treharne H, King S, Henson MC, Schneider SA (eds). Springer-Verlag, 299–318.
[28]
Dadeau F, de Kermadec A, Tissot R. Combining scenario- and model-based testing to ensure POSIX compliance. Abz'2008, International Conference on ASM, B and Z, volume 5238 of LNCS, London, United Kingdom, September 2008; 153–166. Springer.
[29]
Masson P-A, Potet M-L, Julliand J, Tissot R, Debois G, Legeard B, Chetali B, Bouquet F, Jaffuel E, Van Aertrick L, Andronick J, Haddad A. An access control model based testing approach for smart card applications: results of the POSÉ project. JIAS, Journal of Information Assurance and Security 2010; 5(1): 335–351.
[30]
Common criteria for information technology security evaluation, version 3.1. Technical Report CCMB-2006-09-001, Sept 2006.
[31]
Wittevrongel J, Maurer F. SCENTOR: scenario-based testing of e-business applications. Wetice '01: Proceedings of the 10th IEEE International Workshops on Enabling Technologies, Washington, DC, USA, 2001; 41–48. IEEE Computer Society.
[32]
Ryser J, Glinz M. A practical approach to validating and testing software systems using scenarios. Quality Week Europe, Brussels, 1999.
[33]
Tsai WT, Yu L, Saimi A. Scenario-based object-oriented test frameworks for testing distributed systems. Future Trends of Distributed Computing Systems, IEEE International Workshop 2003; 0: 288.
[34]
Binder RV. Testing Object-oriented Systems: Models, Patterns, and Tools. Addison-Wesley Longman Publishing Co., Inc.: Boston, MA, USA, 1999.
[35]
Auguston M, Michael JB, Shing M-T. Environment behavior models for scenario generation and testing automation. A-most '05: Proceedings of the 1st international workshop on advances in model-based testing, New York, NY, USA, 2005; 1–6. ACM.
[36]
Clarke D, Jéron T, Rusu V, Zinovieva E. STG: a tool for generating symbolic test programs and oracles from operational specifications. Esec/fse-9: Proceedings of the 8th european software engineering conference held jointly with 9th ACM SIGSOFT international symposium on foundations of software engineering, New York, NY, USA, 2001; 301–302. ACM.
[37]
Maury O, Ledru Y, du Bousquet L. Intégration de TOBIAS et UCASTING pour la génération de tests, 16th International Conference Software and Systems and their applications-ICSSEA,2003.
[38]
van Aertryck L, Benveniste M, Le Métayer D. Casting: a formally based software test generation method. ICFEM '97: Proceedings of the 1st international conference on formal engineering methods, Washington, DC, USA, 1997; 101. IEEE Computer Society.
[39]
Tillmann N, Schulte W. Parameterized unit tests. SIGSOFT Softw. Eng. Notes 2005; 30(5): 253–262.
[40]
The SpecExplorer 2010 web site, 2010. http://msdn.microsoft.com/en-us/devlabs/ee692301.aspx.
[41]
Gargantini A, Heitmeyer C. Using model checking to generate tests from requirements specifications. SIGSOFT Softw. Eng. Notes 1999; 24(6): 146–162.
[42]
Tan L, Sokolsky O, Lee I. Specification-based testing with linear temporal logic. Iri'2004, ieee int. conf. on information reuse and integration, November 2004; 413–498.
[43]
Fraser G, Wotawa F. Using model-checkers to generate and analyze property relevant test-cases. Software Quality Journal 2008; 16: 161–183.
[44]
Masson P-A, Julliand J, Plessis J-C, Jaffuel E, Debois G. Automatic generation of model based tests for a class of security properties. In A-most'07, 3rd int. workshop on advances in model based testing: London, UK. ACM Press, July 2007; 12–22.
[45]
Falcone Y, Mounier L, Fernandez J-C, Richier J-L. j-POST: a Java Toolchain for property-oriented software testing. Electr. Notes Theor. Comput. Sci. 2008; 220(1): 29–41.
[46]
Rusu V, Marchand H, Jéron T. Automatic verification and conformance testing for validating safety properties of reactive systems. In Formal methods 2005 (fm05), Vol. 3582, Fitzgerald J, Tarlecki A, Hayes I (eds), LNCS. Springer, 2005 July; 189–204.
[47]
Warmer J, Kleppe A. The Object Constraint Language Second Edition: Getting Your Models Ready for MDA. Addison-Wesley, 2003.
[48]
Butler M, Hallerstede S. The rodin formal modelling tool. BCS-FACS Christmas 2007 Meeting - Formal Methods In Industry, London. 2007 December.
Recommendations
jSynoPSys -- A Scenario-Based Testing Tool based on the Symbolic Animation of B Machines
This paper presents the jSynoPSys tool that implements the concept of Scenario-Based Testing from B machines. This consists in describing execution scenarios, expressed as regular expressions over the operations of the system, coupled with intermediate ...
Scenario-Based Test Case Generation Using Event-B Models
VALID '09: Proceedings of the 2009 First International Conference on Advances in System Testing and Validation LifecycleIn this paper we present an extension of the previously reported model-based testing approach that is based on formal models and user-provided testing scenarios. In this approach, the user provides a testing scenario on the level of an abstract model. ...
Scenario-based testing from UML/OCL behavioral models
We present in this article a way to produce test suites applied to the POSIX mini-challenge based on a behavioral model of a file system manager written in UML/OCL. We illustrate the limitations of a fully automated test generation approach, which ...
Comments
Information & Contributors
Information
Published In
Publisher
John Wiley and Sons Ltd.
United Kingdom
Publication History
Published: 01 September 2012
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 02 Sep 2024
Other Metrics
Citations
Cited By
View all- khan Miftikhar SIqbal MSherin S(2018)Empirical studies omit reporting necessary detailsComputer Standards & Interfaces10.1016/j.csi.2017.08.00255:C(156-170)Online publication date: 1-Jan-2018
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in