research-article

Generating test cases for specification mining

Authors:

Valentin Dallmeier,

Christoph Mallon,

Sebastian Hack,

Andreas ZellerAuthors Info & Claims

ISSTA '10: Proceedings of the 19th international symposium on Software testing and analysis

Pages 85 - 96

https://doi.org/10.1145/1831708.1831719

Published: 12 July 2010 Publication History

Abstract

Dynamic specification mining observes program executions to infer models of normal program behavior. What makes us believe that we have seen sufficiently many executions? The typestate miner generates test cases that cover previously unobserved behavior, systematically extending the execution space and enriching the specification. To our knowledge, this is the first combination of systematic test case generation and typestate mining--a combination with clear benefits: On a sample of 800 defects seeded into six Java subjects, a static typestate verifier fed with enriched models would report significantly more true positives, and significantly fewer false positives than the initial models.

References

[1]

1} G. Ammons, R. Bod1k, and J. Larus. Mining specifications. In Conference Record of POPL'02: The 29th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 4--16, Portland, Oregon, Jan. 16--18, 2002.

Digital Library

[2]

M. Barnett, R. Deline, M. Fähndrich, B. Jacobs, K. R. Leino, W. Schulte, and H. Venter. The Spec# programming system: Challenges and directions. pages 144--152, 2008.

[3]

A. Bertolino, P. Inverardi, P. Pelliccione, and M. Tivoli. Automatic synthesis of behavior protocols for composable web-services. In ESEC/FSE '09: Proceedings of the the 7th joint meeting of the European software engineering Conference and the ACM SIGSOFT symposium on The foundations of software engineering, pages 141--150, New York, NY, USA, 2009. ACM.

Digital Library

[4]

K. Bierhoff and J. Aldrich. Modular typestate checking of aliased objects. In OOPSLA '07: Proceedings of the 22nd annual ACM SIGPLAN Conference on Object-oriented programming systems and applications, pages 301--320, New York, NY, USA, 2007. ACM.

Digital Library

[5]

I. Ciupa, A. Leitner, M. Oriol, and B. Meyer. Experimental assessment of random testing for object-oriented software. In ISSTA '07: Proceedings of

Digital Library

[6]

the 2007 International symposium on Software testing and analysis, pages 84--94, New York, NY, USA, 2007. ACM.

[7]

J. Cook and A. Wolf. Discovering Models of Software Processes from Event-Based Data. ACM Transactions on Software Engineering and Methodology, 7(3):215--249, July 1998.

Digital Library

[8]

V. Dallmeier, C. Lindig, A. Wasylkowski, and A. Zeller. Mining object behavior with ADABU. In WODA 2006: ICSE Workshop on Dynamic Analysis, May 2006.

Digital Library

[9]

V. Dallmeier, A. Zeller, and B. Meyer. Generating fixes from object behavior anomalies. In Proceedings of the 24th IEEE/ACM International Conference on Automated Software Engineering, Auckland, New Zealand, November 2009.

Digital Library

[10]

R. Deline and M. Fahndrich. Typestates for objects. In In Proc. 18th ECOOP, pages 465--490. Springer, 2004.

[11]

M. D. Ernst, J. Cockrell, W. G. Griswold, and D. Notkin. Dynamically discovering likely program invariants to support program evolution. IEEE Transactions on Software Engineering, 27(2):1--25, Feb. 2001. A previous version appeared in ICSE '99, Proceedings of the 21st International Conference on Software Engineering, pages 213--224, Los Angeles, CA, USA, May 19--21, 1999.

Digital Library

[12]

S. J. Fink, E. Yahav, N. Dor, G. Ramalingam, and E. Geay. Effective typestate verification in the presence of aliasing. ACM Transactions Software Engineering Methodology, 17(2):1--34, 2008.

Digital Library

[13]

E. Geay, E. Yahav, and S. Fink. Continuous code--quality assurance with SAFE. In PEPM '06: Proceedings of the 2006 ACM SIGPLAN symposium on Partial evaluation and semantics-based program manipulation, pages 145--149, New York, NY, USA, 2006. ACM Press.

Digital Library

[14]

E. Gold. Language identification in the limit. Information and Control, pages 447--474, 1967.

[15]

N. Gupta and Z. V. Heidepriem. A new structural coverage criterion for dynamic detection of program invariants. Automated Software Engineering, International Conference on, 0:49, 2003.

[16]

R. M. Hierons, K. Bogdanov, J. P. Bowen, R. Cleaveland, J. Derrick, J. Dick, M. Gheorghe, M. Harman, K. Kapoor, P. Krause, and G. Luttgen. Using formal specifications to support testing. ACM Comput. Surv., 41(2):1--76, 2009.

Digital Library

[17]

http://ostatic.com/ristretto. Ristretto 1.0, 2010. {17} J. C. King. Symbolic execution and program testing.

[18]

Commun. ACM, 19(7):385--394, 1976. {18} D. Lorenzoli, L. Mariani, and M. Pezz`e. Automatic

Digital Library

[19]

generation of software behavioral models. In ICSE '08: Proceedings of the 30th International Conference on Software engineering, pages 501--510, New York, NY, USA, 2008. ACM.

Digital Library

[20]

R. Majumdar and K. Sen. Hybrid concolic testing. In

[21]

ICSE '07: Proceedings of the 29th International Conference on Software Engineering, pages 416--426, Washington, DC, USA, 2007. IEEE Computer Society.

[22]

P. Mcminn. Search-based software test data generation: A survey. Software Testing, Verification and Reliability, 14:105--156, 2004.

[23]

A. Mesbah and A. van Deursen. Invariant-based automatic testing of AJAX user interfaces. In ICSE '09: Proceedings of the 2009 IEEE 31st International Conference on Software Engineering, pages 210--220, Washington, DC, USA, 2009. IEEE Computer Society.

Digital Library

[24]

B. Meyer, A. Fiva, I. Ciupa, A. Leitner, Y. Wei, and E. Stapf. Programs that test themselves. Computer, 42:46--55, 2009.

Digital Library

[25]

A. Milicevic, S. Misailovic, D. Marinov, and S. Khurshid. Korat: A tool for generating structurally complex test inputs. In ICSE '07: Proceedings of the 29th International Conference on Software Engineering, pages 771--774, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[26]

M. K. Ramanathan, A. Grama, and S. Jagannathan. Static specification inference using predicate mining. In PLDI '07: Proceedings of the 2007 ACM SIGPLAN Conference on Programming language design and implementation, pages 123--134, New York, 2007. ACM.

Digital Library

[27]

S. Shoham, E. Yahav, S. Fink, and M. Pistoia. Static specification mining using automata-based abstractions. In ISSTA '07: of the 2007 int. symposium on Software testing and analysis, pages 174--184, New York, 2007. ACM.

Digital Library

[28]

M.SridharanandR.Bod1k.Refinement-based context-sensitive points--to analysis for java. SIGPLAN Not., 41(6):387--400, 2006.

Digital Library

[29]

R. E. Strom and S. Yemini. Typestate: A programming language concept for enhancing software reliability. Transactions Software Engineering, 12(1):157--171, 1986.

Digital Library

[30]

P. Tonella. Evolutionary testing of classes. SIGSOFT Softw. Eng. Notes, 29(4):119--128, 2004.

Digital Library

[31]

M. Veanes, C. Campbell, W. Schulte, and N. Tillmann. Online testing with model programs. SIGSOFT Software Engineering Notes, 30(5):273--282, 2005.

Digital Library

[32]

A. Wasylkowski, A. Zeller, and C. Lindig. Detecting object usage anomalies. In ESEC--FSE'07: Proceedings of the the 6th joint meeting of the European software engineering Conference and the ACM SIGSOFT symposium on the foundations of software engineering, pages 35--44, New York, NY, USA, 2007. ACM.

Digital Library

[33]

J. Whaley, M. C. Martin, and M. S. Lam. Automatic extraction of object-oriented component interfaces. In ISSTA '02: Proceedings of the 2002 ACM SIGSOFT International symposium on Software testing and analysis, pages 218--228, New York, NY, USA, 2002.

Digital Library

[34]

T. Xie and D. Notkin. Mutually enhancing test generation and specification inference. In Proc. 3rd International Workshop on Formal Approaches to Testing of Software (FATES 03), volume 2931 of LNCS, pages 60--69, October 2003.

Cited By

Brun YLin TSomerville JMyers EEbner N(2023)Blindspots in Python and Java APIs Result in Vulnerable CodeACM Transactions on Software Engineering and Methodology10.1145/357185032:3(1-31)Online publication date: 26-Apr-2023
https://dl.acm.org/doi/10.1145/3571850
Clerissi DDi Rocco JDi Ruscio DDi Sipio CIhirwe FMariani LMicucci DRossi MRubei R(2023)Supporting Early-Safety Analysis of IoT Systems by Exploiting Testing Techniques2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00089(520-529)Online publication date: 1-Oct-2023
https://doi.org/10.1109/MODELS-C59198.2023.00089
Yan AZhong HSong DJia L(2023)How do programmers fix bugs as workarounds? An empirical study on Apache projectsEmpirical Software Engineering10.1007/s10664-023-10318-728:4Online publication date: 28-Jun-2023
https://dl.acm.org/doi/10.1007/s10664-023-10318-7
Show More Cited By

Index Terms

Generating test cases for specification mining

Recommendations

Adversarial Specification Mining
Continuous Special Section: AI and SE

There have been numerous studies on mining temporal specifications from execution traces. These approaches learn finite-state automata (FSA) from execution traces when running tests. To learn accurate specifications of a software system, many tests are ...
Automatically Generating Test Cases for Specification Mining

Dynamic specification mining observes program executions to infer models of normal program behavior. What makes us believe that we have seen sufficiently many executions? The TAUTOKO (“Tautoko” is the Mãori word for “enhance, enrich.”) typestate miner ...
Achieving Effective Test Suites for Reactive Systems using Specification Mining and Test Suite Reduction Techniques

Failures in reactive embedded systems are often unacceptable. Moreover, effective testing of such systems to detect potential critical failures is a difficult task.

We present an automated black box test suite generation technique for reactive systems. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA '10: Proceedings of the 19th international symposium on Software testing and analysis

July 2010

294 pages

ISBN:9781605588230

DOI:10.1145/1831708

General Chair:
Paolo Tonella
Fondazione Bruno Kessler -- IRST, Italy
,
Program Chair:
Alessandro Orso
Georgia Institute of Technology, USA

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '10

Sponsor:

SIGSOFT

ISSTA '10: International Symposium on Software Testing and Analysis

July 12 - 16, 2010

Trento, Italy

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

116
Total Citations
View Citations
836
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)5

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Brun YLin TSomerville JMyers EEbner N(2023)Blindspots in Python and Java APIs Result in Vulnerable CodeACM Transactions on Software Engineering and Methodology10.1145/357185032:3(1-31)Online publication date: 26-Apr-2023
https://dl.acm.org/doi/10.1145/3571850
Clerissi DDi Rocco JDi Ruscio DDi Sipio CIhirwe FMariani LMicucci DRossi MRubei R(2023)Supporting Early-Safety Analysis of IoT Systems by Exploiting Testing Techniques2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00089(520-529)Online publication date: 1-Oct-2023
https://doi.org/10.1109/MODELS-C59198.2023.00089
Yan AZhong HSong DJia L(2023)How do programmers fix bugs as workarounds? An empirical study on Apache projectsEmpirical Software Engineering10.1007/s10664-023-10318-728:4Online publication date: 28-Jun-2023
https://dl.acm.org/doi/10.1007/s10664-023-10318-7
Yang NCuijpers PHendriks DSchiffelers RLukkien JSerebrenik A(2023)An interview study about the use of logs in embedded software engineeringEmpirical Software Engineering10.1007/s10664-022-10258-828:2Online publication date: 11-Feb-2023
https://dl.acm.org/doi/10.1007/s10664-022-10258-8
Wang YWu RWang CWen MLiu YCheung SYu HXu CZhu Z(2022)Will Dependency Conflicts Affect My Program's Semantics?IEEE Transactions on Software Engineering10.1109/TSE.2021.305776748:7(2295-2316)Online publication date: 1-Jul-2022
https://doi.org/10.1109/TSE.2021.3057767
Zhong HWang XMei H(2022)Inferring Bug Signatures to Detect Real BugsIEEE Transactions on Software Engineering10.1109/TSE.2020.299697548:2(571-584)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TSE.2020.2996975
Cabral LMiranda BLima Id'Amorim M(2022)RVprio: A tool for prioritizing runtime verification violationsSoftware Testing, Verification and Reliability10.1002/stvr.181332:5Online publication date: 7-Mar-2022
https://doi.org/10.1002/stvr.1813
Schoofs EAbdi MDemeyer S(2022)AmPyfier: Test amplification in PythonJournal of Software: Evolution and Process10.1002/smr.249034:11Online publication date: 24-Jul-2022
https://doi.org/10.1002/smr.2490
Kang HLo D(2021)Adversarial Specification MiningACM Transactions on Software Engineering and Methodology10.1145/342430730:2(1-40)Online publication date: 3-Jan-2021
https://dl.acm.org/doi/10.1145/3424307
Afzal AMotwani MStolee KBrun YLe Goues C(2021)SOSRepair: Expressive Semantic Search for Real-World Program RepairIEEE Transactions on Software Engineering10.1109/TSE.2019.294491447:10(2162-2181)Online publication date: 1-Oct-2021
https://doi.org/10.1109/TSE.2019.2944914
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents