research-article

Interpolated n-grams for model based testing

Authors:

Roberto Tiella,

Cu Duy NguyenAuthors Info & Claims

ICSE 2014: Proceedings of the 36th International Conference on Software Engineering

Pages 562 - 572

https://doi.org/10.1145/2568225.2568242

Published: 31 May 2014 Publication History

Abstract

Models - in particular finite state machine models - provide an invaluable source of information for the derivation of effective test cases. However, models usually approximate part of the program semantics and capture only some of the relevant dependencies and constraints. As a consequence, some of the test cases that are derived from models are infeasible.

In this paper, we propose a method, based on the computation of the N-gram statistics, to increase the likelihood of deriving feasible test cases from a model. Correspondingly, the level of model coverage is also expected to increase, because infeasible test cases do not contribute to coverage. While N-grams do improve existing test case derivation methods, they show limitations when the N-gram statistics is incomplete, which is expected to necessarily occur as N increases. Interpolated N-grams overcome such limitation and show the highest performance of all test case derivation methods compared in this work.

References

[1]

A. Andrews, J. Offutt, and R. Alexander. Testing Web Applications by Modeling with FSMs. Software and System Modeling, Vol 4, n. 3, pages 326–345, 2005.

[2]

Anneliese Amschler Andrews, Jeff Offutt, and Roger T. Alexander. Testing web applications by modeling with fsms. Software and System Modeling, 4(3):326–345, 2005.

Digital Library

[3]

A. Biermann and J. Feldman. On the synthesis of finite-state machines from samples of their behavior. IEEE Trans. on Computers, 21(6), 1972.

Digital Library

[4]

Lionel C. Briand and Yvan Labiche. A UML-based approach to system testing. In Proceedings of the 4th International Conference on The Unified Modeling Language, Modeling Languages, Concepts, and Tools, pages 194–208. Springer-Verlag, 2001.

Digital Library

[5]

James C. Corbett, Matthew B. Dwyer, John Hatcliff, Shawn Laubach, Corina S. Pasareanu, Robby, and Hongjun Zheng. Bandera: Extracting finite-state models from java source code. In Proceedings of the International Conference on Software Engineering, pages 439–448, 2000.

Digital Library

[6]

Valentin Dallmeier, Nikolai Knopp, Christoph Mallon, Sebastian Hack, and Andreas Zeller. Generating test cases for specification mining. In Proceedings of the 19th international symposium on Software testing and analysis, ISSTA ’10, pages 85–96. ACM, 2010.

Digital Library

[7]

Valentin Dallmeier, Christian Lindig, Andrzej Wasylkowski, and Andreas Zeller. Mining object behavior with ADABU. In Proceedings of the 2006 international workshop on Dynamic systems analysis, pages 17–24, 2006.

Digital Library

[8]

Valentin Dallmeier, Christian Lindig, Andrzej Wasylkowski, and Andreas Zeller. Mining object behavior with ADABU. In Proc. of the International Workshop on Dynamic Analysis (WODA), pages 17–24, Shangai, China, May 2006.

Digital Library

[9]

Arilo C. Dias Neto, Rajesh Subramanyan, Marlon Vieira, and Guilherme H. Travassos. A survey on model-based testing approaches: a systematic review. In Proc. of the International Workshop on Empirical Assessment of Software Engineering Languages and Technologies (co-located with ASE’07), WEASELTech ’07, pages 31–36. ACM, 2007.

Digital Library

[10]

Daniel Jurafsky and James H. Martin. Speech and Language Processing: An introduction to speech recognition, computational linguistics and natural language processing. Pearson Prentice Hall, 2007.

Digital Library

[11]

Y.G. Kim, H.S. Hong, D.H. Bae, and S.D. Cha. Test cases generation from UML state diagrams. Software, IEE Proceedings -, 146(4):187 –192, aug 1999.

[12]

Ivo Krka, Yuriy Brun, Daniel Popescu, Joshua Garcia, and Nenad Medvidovic. Using dynamic execution traces and program invariants to enhance behavioral model inference. In ICSE (2), pages 179–182, 2010.

Digital Library

[13]

Davide Lorenzoli, Leonardo Mariani, and Mauro Pezzè. Automatic generation of software behavioral models. In 30th International Conference on Software Engineering (ICSE). IEEE Computer Society, May 2008.

Digital Library

[14]

Chris Manning and Hinrich Sch utze. Foundations of Statistical Natural Language Processing. MIT Press, 1999.

Digital Library

[15]

Alessandro Marchetto, Paolo Tonella, and Filippo Ricca. State-based testing of ajax web applications. In Proc. of IEEE International Conference on Software Testing (ICST), pages 121–131, April 2008.

Digital Library

[16]

A. Mesbah, A. van Deursen, and D. Roest. Invariant-based automatic testing of modern web applications. IEEE Transactions on Software Engineering, 38(1):35–53, 2012.

Digital Library

[17]

Cu D. Nguyen, Alessandro Marchetto, and Paolo Tonella. Combining model-based and combinatorial testing for effective test case generation. In Proc of the ACM International Symposium on Software Testing and Analysis (ISSTA), pages 100–110, 2012.

Digital Library

[18]

Jeff Offutt and Aynur Abdurazik. Generating tests from UML specifications. In Robert France and Bernhard Rumpe, editors, UML 99 : The Unified Modeling Language, volume 1723 of Lecture Notes in Computer Science, pages 76–76. Springer Berlin / Heidelberg, 1999.

Digital Library

[19]

Mauro Pezzè and Michal Young. Software Testing and Analysis: Process, Principles and Techniques. John Wiley and Sons, 2007.

Digital Library

[20]

Muhammad Shafique and Yvan Labiche. A systematic review of model based testing tool support. Technical Report Technical Report SCE-10-04, Carleton University, Canada, May 2010.

[21]

Phil Stocks and David Carrington. A framework for specification-based testing. IEEE Trans. Softw. Eng., 22:777–793, November 1996.

Digital Library

[22]

Paolo Tonella, Alessandro Marchetto, Cu D. Nguyen, Yue Jia, Kiran Lakhotia, and Mark Harman. Finding the optimal balance between over and under approximation of models inferred from execution logs. In Proc of the Fifth IEEE International Conference on Software Testing, Verification and Validation (ICST), pages 21–30, 2012.

Digital Library

[23]

Paolo Tonella, Cu D. Nguyen, Alessandro Marchetto, Kiran Lakhotia, and Mark Harman. Automated generation of state abstraction functions using data invariant inference. In Proceedings of the 8th International Workshop on Automation of Software Test (AST), 2013.

Digital Library

[24]

Paolo Tonella, Roberto Tiella, and Cu D. Nguyen. N-gram based test sequence generation from finite state models. In Proceedings of the 1st Future Internet Workshop (FITTEST), 2013.

[25]

C. D. Turner and D. J. Robson. The state-based testing of object-oriented programs. In Proc. of the Conference on Software Maintenance (ICSM), pages 302–310, Montreal, Canada, September 1993. IEEE Computer Society.

Digital Library

[26]

N. Walkinshaw, K. Bogdanov, C. Damas, B. Lambeau, and P. Dupont. A framework for the competitive evaluation of model inference techniques. In Proceedings of the International Workshop on Model Inference in Testing (MIIT), 2010.

Digital Library

[27]

Xun Yuan and Atif M. Memon. Using GUI run-time state as feedback to generate test cases. In Proc. the International Conference on Software Engineering (ICSE), pages 396–405, Washington, DC, USA, May 23–25, 2007. IEEE Computer Society.

Digital Library

Cited By

Guizzo GCalifano FSarro FFerrucci FHarman M(2023)Inferring test models from user bug reports using multi-objective searchEmpirical Software Engineering10.1007/s10664-023-10333-828:4Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1007/s10664-023-10333-8
Rao CLi NLei YGuo JZhang YKacker RKuhn D(2022)Combinatorial Test Generation for Multiple Input Models With Shared ParametersIEEE Transactions on Software Engineering10.1109/TSE.2021.306595048:7(2606-2628)Online publication date: 1-Jul-2022
https://doi.org/10.1109/TSE.2021.3065950
Leveau JBlanc XRéveillère LFalleri JRouvoy R(2022)Fostering the diversity of exploratory testing in web applicationsSoftware Testing, Verification and Reliability10.1002/stvr.182732:5Online publication date: 19-Jun-2022
https://doi.org/10.1002/stvr.1827
Show More Cited By

Index Terms

Interpolated n-grams for model based testing
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Assessing model-based testing: an empirical study conducted in industry
ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software Engineering

We compare manual testing without any automation performed by a tester at a software company with model-based testing (MBT) performed by a tester at a research center.

The system under test (SUT), of which two different versions were tested by each ...
Model-in-the-Loop Testing for Cyber Physical Systems

Context: Nowadays, there is of high interest to use automated testing, not only because it optimizes the manual testing by reducing the needed time and cost, but also because manual testing is more likely to produce errors. Increasing the safety of ...
An Evaluation of Model Checkers for Specification Based Test Case Generation
ICST '09: Proceedings of the 2009 International Conference on Software Testing Verification and Validation

Under certain constraints the test case generation problem can be represented as a model checking problem, thus enabling the use of powerful model checking tools to perform the test case generation automatically. There are, however, several different ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE 2014: Proceedings of the 36th International Conference on Software Engineering

May 2014

1139 pages

ISBN:9781450327565

DOI:10.1145/2568225

General Chair:
Pankaj Jalote
IIIT-Delhi, India
,
Program Chairs:
Lionel Briand
University of Luxembourg, Luxembourg
,
André van der Hoek
University of California, Irvine, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE '14

Sponsor:

SIGSOFT

ICSE '14: 36th International Conference on Software Engineering

May 31 - June 7, 2014

Hyderabad, India

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
386
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Guizzo GCalifano FSarro FFerrucci FHarman M(2023)Inferring test models from user bug reports using multi-objective searchEmpirical Software Engineering10.1007/s10664-023-10333-828:4Online publication date: 20-Jun-2023
https://dl.acm.org/doi/10.1007/s10664-023-10333-8
Rao CLi NLei YGuo JZhang YKacker RKuhn D(2022)Combinatorial Test Generation for Multiple Input Models With Shared ParametersIEEE Transactions on Software Engineering10.1109/TSE.2021.306595048:7(2606-2628)Online publication date: 1-Jul-2022
https://doi.org/10.1109/TSE.2021.3065950
Leveau JBlanc XRéveillère LFalleri JRouvoy R(2022)Fostering the diversity of exploratory testing in web applicationsSoftware Testing, Verification and Reliability10.1002/stvr.182732:5Online publication date: 19-Jun-2022
https://doi.org/10.1002/stvr.1827
Fortz SMousavi MSchobbens P(2021)LIFTSProceedings of the 25th ACM International Systems and Software Product Line Conference - Volume B10.1145/3461002.3473066(1-6)Online publication date: 6-Sep-2021
https://dl.acm.org/doi/10.1145/3461002.3473066
Leveau JBlanc XReveillere LFalleri JRouvoy R(2020)Fostering the Diversity of Exploratory Testing in Web Applications2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)10.1109/ICST46399.2020.00026(164-174)Online publication date: Oct-2020
https://doi.org/10.1109/ICST46399.2020.00026
Namiot DRomanov V(2020)On Data Analysis of Software RepositoriesConvergent Cognitive Information Technologies10.1007/978-3-030-37436-5_24(263-272)Online publication date: 1-Feb-2020
https://doi.org/10.1007/978-3-030-37436-5_24
Derakhshanfar PDevroey XPerrouin GZaidman Avan Deursen A(2020)Search‐based crash reproduction using behavioural model seedingSoftware Testing, Verification and Reliability10.1002/stvr.173330:3Online publication date: 12-Apr-2020
https://doi.org/10.1002/stvr.1733
Tran NTran HNguyen SNguyen HNguyen TGuéhéneuc YKhomh FSarro F(2019)Does BLEU score work for code migration?Proceedings of the 27th International Conference on Program Comprehension10.1109/ICPC.2019.00034(165-176)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICPC.2019.00034
Lin JJabbarvand RMalek SZimmermann TLawall JMarinov D(2019)Test transfer across mobile apps through semantic mappingProceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE.2019.00015(42-53)Online publication date: 10-Nov-2019
https://dl.acm.org/doi/10.1109/ASE.2019.00015
Jiang SWang HZhang YXue MQian JZhang M(2019)An Approach for Detecting Infeasible Paths Based on a SMT SolverIEEE Access10.1109/ACCESS.2019.29185587(69058-69069)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2918558
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents