Article

Check 'n' crash: combining static checking and testing

Authors:

Christoph Csallner,

Yannis SmaragdakisAuthors Info & Claims

ICSE '05: Proceedings of the 27th international conference on Software engineering

Pages 422 - 431

https://doi.org/10.1145/1062455.1062533

Published: 15 May 2005 Publication History

Abstract

We present an automatic error-detection approach that combines static checking and concrete test-case generation. Our approach consists of taking the abstract error conditions inferred using theorem proving techniques by a static checker (ESC/Java), deriving specific error conditions using a constraint solver, and producing concrete test cases (with the JCrasher tool) that are executed to determine whether an error truly exists. The combined technique has advantages over both static checking and automatic testing individually. Compared to ESC/Java, we eliminate spurious warnings and improve the ease-of-comprehension of error reports through the production of Java counterexamples. Compared to JCrasher, we eliminate the blind search of the input space, thus reducing the testing time and increasing the test quality.

References

[1]

K. Beck and E. Gamma. Test infected: Programmers love writing tests. Java Report, 3(7):37--50, July 1998.]]

[2]

D. Beyer, A. J. Chlipala, T. A. Henzinger, R. Jhala, and R. Majumdar. Generating tests from counterexamples. In Proceedings of the 26th International Conference on Software Engineering, pages 326--335. IEEE Computer Society Press, May 2004.]]

Digital Library

[3]

C. Boyapati, S. Khurshid, and D. Marinov. Korat: Automated testing based on Java predicates. In Proceedings of the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 123--133. ACM Press, July 2002.]]

Digital Library

[4]

D. R. Cok and J. R. Kiniry. ESC/Java2: Uniting ESC/Java and JML: Progress and issues in building and using ESC/Java2. Technical Report NIII-R0413, Nijmegen Institute for Computing and Information Science, May 2004.]]

[5]

C. Csallner and Y. Smaragdakis. JCrasher: An automatic robustness tester for Java. Software---Practice & Experience, 34(11):1025--1050, Sept. 2004.]]

Digital Library

[6]

D. Detlefs, G. Nelson, and J. B. Saxe. Simplify: A theorem prover for program checking. Technical Report HPL-2003-148, Hewlett-Packard Systems Research Center, July 2003.]]

[7]

M. D. Ernst, J. Cockrell, W. G. Griswold, and D. Notkin. Dynamically discovering likely program invariants to support program evolution. In Proceedings of the 21st International Conference on Software Engineering, pages 213--224. IEEE Computer Society Press, May 1999.]]

Digital Library

[8]

C. Flanagan and K. R. M. Leino. Houdini, an annotation assistant for ESC/Java. In J. N. Oliveira and P. Zave, editors, FME 2001: Formal Methods for Increasing Software Productivity: International Symposium of Formal Methods Europe, pages 500--517. Springer, Mar. 2001.]]

Digital Library

[9]

C. Flanagan, K. R. M. Leino, M. Lillibridge, G. Nelson, J. B. Saxe, and R. Stata. Extended static checking for Java. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation, pages 234--245. ACM Press, June 2002.]]

Digital Library

[10]

S. Hallem, B. Chelf, Y. Xie, and D. Engler. A system and language for building system-specific, static analyses. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation, pages 69--82. ACM Press, June 2002.]]

Digital Library

[11]

M. Hapner, R. Burridge, R. Sharma, and J. Fialli. Java message service: Version 1.1. Sun Microsystems, Inc., Apr. 2002.]]

[12]

E. Hatcher and S. Loughran. Java development with Ant. Manning Publications, Aug. 2002.]]

Digital Library

[13]

D. Hovemeyer and W. Pugh. Finding bugs is easy. In Companion to the 19th annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 132--136. ACM Press, Oct. 2004.]]

Digital Library

[14]

D. Jackson and M. Vaziri. Finding bugs with a constraint solver. In M. J. Harrold, editor, Proceedings of the 2000 ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 14--25. ACM Press, Aug. 2000.]]

Digital Library

[15]

T. Kremenek and D. Engler. Z-ranking: Using statistical analysis to counter the impact of static analysis approximations. In R. Cousot, editor, Static Analysis: 10th Annual International Static Analysis Symposium, pages 295--315. Springer, June 2003.]]

Digital Library

[16]

D. Kroening, A. Groce, and E. M. Clarke. Counterexample guided abstraction refinement via program execution. In J. Davies, W. Schulte, and M. Barnett, editors, Formal Methods and Software Engineering: 6th International Conference on Formal Engineering Methods, pages 224--238. Springer, Nov. 2004.]]

[17]

G. T. Leavens, A. L. Baker, and C. Ruby. Preliminary design of JML: A behavioral interface specification language for Java. Technical Report TR98-06y, Department of Computer Science, Iowa State University, June 1998.]]

[18]

K. R. M. Leino. Efficient weakest preconditions. Technical Report MSR-TR-2004-34, Microsoft Research, Apr. 2004.]]

[19]

K. R. M. Leino, G. Nelson, and J. B. Saxe. ESC/Java user's manual. Technical Report 2000-002, Compaq Computer Corporation Systems Research Center, Oct. 2000.]]

[20]

M. Musuvathi and D. Engler. Some lessons from using static analysis and software model checking for bug finding. In B. Cook, S. Stoller, and W. Visser, editors, SoftMC 2003: Workshop on Software Model Checking. Elsevier, July 2003.]]

[21]

N. Rutar, C. B. Almazan, and J. S. Foster. A comparison of bug finding tools for Java. In Proceedings of the 15th International Symposium on Software Reliability Engineering (ISSRE'04), pages 245--256. IEEE Computer Society Press, Nov. 2004.]]

Digital Library

[22]

H. Schlenker and G. Ringwelski. POOC: A platform for object-oriented constraint programming. In B. O'Sullivan, editor, Recent Advances in Constraints: Joint ERCIM/CologNet International Workshop on Constraint Solving and Constraint Logic Programming, pages 159--170. Springer, June 2002.]]

Digital Library

[23]

B. Steensgaard. Points-to analysis in almost linear time. In Proceedings of the 23rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 32--41. ACM Press, Jan. 1996.]]

Digital Library

[24]

D. Stotts, M. Lindsey, and A. Antley. An informal formal method for systematic JUnit test case generation. In D. Wells and L. A. Williams, editors, Proceedings of the Second XP Universe and First Agile Universe Conference on Extreme Programming and Agile Methods - XP/Agile Universe 2002, pages 131--143. Springer, Aug. 2002.]]

Digital Library

[25]

M. Vaziri and D. Jackson. Checking properties of heap-manipulating procedures with a constraint solver. In H. Garavel and J. Hatcliff, editors, Tools and Algorithms for the Construction and Analysis of Systems: 9th International Conference, pages 505--520. Springer, Apr. 2003.]]

Digital Library

[26]

T. Xie and D. Notkin. Tool-assisted unit test selection based on operational violations. In Proceedings of the 18th Annual International Conference on Automated Software Engineering (ASE 2003), pages 40--48. IEEE Computer Society Press, Oct. 2003.]]

[27]

Y. Xie and D. Engler. Using redundancies to find errors. IEEE Transactions on Software Engineering, 29(10):915--928, Oct. 2003.]]

Digital Library

Cited By

Haltermann JJakobs MRichter CWehrheim H(2024)Parallel program analysis on path rangesScience of Computer Programming10.1016/j.scico.2024.103154238:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.scico.2024.103154
Haltermann JWehrheim H(2024)Exchanging information in cooperative software validationSoftware and Systems Modeling (SoSyM)10.1007/s10270-024-01155-323:3(695-719)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10270-024-01155-3
He NZhao ZWang JHu YGuo SWang HLiang GLi DChen XGuo YJust RFraser G(2023)Eunomia: Enabling User-Specified Fine-Grained Search in Symbolically Executing WebAssembly BinariesProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598064(385-397)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598064
Show More Cited By

Index Terms

Recommendations

DSD-Crasher: a hybrid analysis tool for bug finding
ISSTA '06: Proceedings of the 2006 international symposium on Software testing and analysis
DSD-Crasher is a bug finding tool that follows a three-step approach to program analysis:
- D. Capture the program's intended execution behavior with dynamic invariant detection. The derived invariants exclude many unwanted values from the program's input ...
DSD-Crasher: A hybrid analysis tool for bug finding

DSD-Crasher is a bug finding tool that follows a three-step approach to program analysis:

D. Capture the program's intended execution behavior with dynamic invariant detection. The derived invariants exclude many unwanted values from the program's input ...
SADAT An Automated Testing Tool

This paper describes the automated testing tool SADAT, which supports the testing of single Fortran modules. The different functions which are integrated in this system are explained, the usage of the tool is demonstrated, and some output results are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '05: Proceedings of the 27th international conference on Software engineering

May 2005

754 pages

ISBN:1581139632

DOI:10.1145/1062455

General Chair:
Gruia-Catalin Roman
Washington University in St. Louis
,
Program Chairs:
William Griswold
University of California, San Diego
,
Bashar Nuseibeh
The Open University, UK

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 May 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICSE05

Sponsor:

ICSE05: 27th International Conference on Software Engineering

May 15 - 21, 2005

MO, St. Louis, USA

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

172
Total Citations
View Citations
1,090
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)3

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Haltermann JJakobs MRichter CWehrheim H(2024)Parallel program analysis on path rangesScience of Computer Programming10.1016/j.scico.2024.103154238:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.scico.2024.103154
Haltermann JWehrheim H(2024)Exchanging information in cooperative software validationSoftware and Systems Modeling (SoSyM)10.1007/s10270-024-01155-323:3(695-719)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10270-024-01155-3
He NZhao ZWang JHu YGuo SWang HLiang GLi DChen XGuo YJust RFraser G(2023)Eunomia: Enabling User-Specified Fine-Grained Search in Symbolically Executing WebAssembly BinariesProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598064(385-397)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598064
Haltermann JJakobs MRichter CWehrheim H(2023)Parallel Program Analysis via Range SplittingFundamental Approaches to Software Engineering10.1007/978-3-031-30826-0_11(195-219)Online publication date: 22-Apr-2023
https://dl.acm.org/doi/10.1007/978-3-031-30826-0_11
Wong WGao RLi YAbreu RWotawa FLi D(2023)Software Fault Localization: an Overview of Research, Techniques, and ToolsHandbook of Software Fault Localization10.1002/9781119880929.ch1(1-117)Online publication date: 21-Apr-2023
https://doi.org/10.1002/9781119880929.ch1
Busse FGharat PCadar CDonaldson ARyu SSmaragdakis Y(2022)Combining static analysis error traces with dynamic symbolic execution (experience paper)Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3533767.3534384(568-579)Online publication date: 18-Jul-2022
https://dl.acm.org/doi/10.1145/3533767.3534384
Liu CWang HLiu TGu DMa YWang HXiao XDwyer MDamian DZeller A(2022)ProMalProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510037(1755-1767)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510037
Alrabaee SDebbabi MWang L(2022)A Survey of Binary Code Fingerprinting Approaches: Taxonomy, Methodologies, and FeaturesACM Computing Surveys10.1145/348686055:1(1-41)Online publication date: 17-Jan-2022
https://dl.acm.org/doi/10.1145/3486860
Beyer DDangl MDietsch DHeizmann MLemberger TTautschnig M(2022)Verification WitnessesACM Transactions on Software Engineering and Methodology10.1145/347757931:4(1-69)Online publication date: 8-Sep-2022
https://dl.acm.org/doi/10.1145/3477579
Tung LTran HLe KHung P(2022)An automated test data generation method for void pointers and function pointers in C/C++ libraries and embedded projectsInformation and Software Technology10.1016/j.infsof.2022.106821145:COnline publication date: 1-May-2022
https://dl.acm.org/doi/10.1016/j.infsof.2022.106821
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents