research-article

Directed Incremental Symbolic Execution

Authors:

Suzette Person,

Sarfraz KhurshidAuthors Info & Claims

ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 24, Issue 1

Article No.: 3, Pages 1 - 42

https://doi.org/10.1145/2629536

Published: 07 October 2014 Publication History

Abstract

The last few years have seen a resurgence of interest in the use of symbolic execution—a program analysis technique developed more than three decades ago to analyze program execution paths. Scaling symbolic execution to real systems remains challenging despite recent algorithmic and technological advances. An effective approach to address scalability is to reduce the scope of the analysis. For example, in regression analysis, differences between two related program versions are used to guide the analysis. While such an approach is intuitive, finding efficient and precise ways to identify program differences, and characterize their impact on how the program executes has proved challenging in practice.

In this article, we present Directed Incremental Symbolic Execution (DiSE), a novel technique for detecting and characterizing the impact of program changes to scale symbolic execution. The novelty of DiSE is to combine the efficiencies of static analysis techniques to compute program difference information with the precision of symbolic execution to explore program execution paths and generate path conditions affected by the differences. DiSE complements other reduction and bounding techniques for improving symbolic execution. Furthermore, DiSE does not require analysis results to be carried forward as the software evolves—only the source code for two related program versions is required. An experimental evaluation using our implementation of DiSE illustrates its effectiveness at detecting and characterizing the effects of program changes.

References

[1]

Saswat Anand, Corina S. Păsăreanu, and Willem Visser. 2009. Symbolic execution with abstraction. Inter. J. Softw. Tools Technol. Transfer 11, 1 (2009), 53--67.

Digital Library

[2]

John Backes, Suzette Person, Neha Rungta, and Oksana Tkachuk. 2013a. Proteus: A change impact analysis framework. Tech. Rep.

Digital Library

[3]

John Backes, Suzette Person, Neha Rungta, and Oksana Tkachuk. 2013b. Regression verification using impact summaries. In Model Checking Software, Springer, 99--116.

[4]

William R. Bush, Jonathan D. Pincus, and David J. Sielaff. 2000. A static analyzer for finding dynamic programming errors. Software: Prac. Exper. 30, 7 (2000), 775--802.

Digital Library

[5]

Cristian Cadar, Daniel Dunbar, and Dawson R. Engler. 2008. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of OSDI. 209--224.

Digital Library

[6]

Cristian Cadar and Dawson R. Engler. 2005. Execution generated test cases: How to make systems code crash itself. In Proceedings of SPIN. 2--23.

Digital Library

[7]

Walter Chochen Chang. 2010. Improving dynamic analysis with data flow analysis. Ph.D. Dissertation, University of Texas at Austin.

[8]

Lori A. Clarke. 1976. A program testing system. In Proceedings of the 1976 Annual Conference (ACM'76). 488--491.

Digital Library

[9]

Christoph Csallner, Nikolai Tillmann, and Yannis Smaragdakis. 2008. DySy: Dynamic symbolic execution for invariant inference. In Proceedings of ICSE. 281--290.

Digital Library

[10]

Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In Proceedings of TACAS. 337--340.

Digital Library

[11]

Xianghua Deng, Robby, and John Hatcliff. 2007. Kiasan/KUnit: Automatic test case generation and analysis feedback for open object-oriented systems. In Proceedings of TAICPART-MUTATION. 3--12.

Digital Library

[12]

Jaco Geldenhuys, Matthew B. Dwyer, and Willem Visser. 2012. Probabilistic symbolic execution. In Proceedings of ISSTA. 166--176.

Digital Library

[13]

Patrice Godefroid. 2007. Compositional Dynamic Test Generation. In Proceedings of POPL. 47--54.

Digital Library

[14]

Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed Automated Random Testing. In Proceedings of PLDI. 213--223.

Digital Library

[15]

Patrice Godefroid, Shuvendu K. Lahiri, and Cindy Rubio-González. 2011. Statically validating must summaries for incremental compositional dynamic test generation. In Proceedings of SAS. 112--128.

Digital Library

[16]

Todd L. Graves, Mary Jean Harrold, Jung-Min Kim, Adam Porter, and Gregg Rothermel. 2001. An empirical study of regression test selection techniques. ACM Trans. Softw. Eng. Meth. 10, 2, 184--208.

Digital Library

[17]

Kobi Inkumsah and Tao Xie. 2008. Improving structural testing of object-oriented programs via integrating evolutionary testing and symbolic execution. In Proceedings of ASE. 297--306.

Digital Library

[18]

Daniel Jackson. 2006. Software Abstractions: Logic, Language, and Analysis. The MIT Press, Cambridge, MA.

Digital Library

[19]

Anjali Joshi and Mats Per Erik Heimdahl. 2005. Model-based safety analysis of simulink models using SCADE design verifier. In Proceedings of SAFECOMP. Lecture Notes in Computer Science, vol. 3688, 122--135.

Digital Library

[20]

Sarfraz Khurshid, Iván García, and Yuk Lai Suen. 2005. Repairing structurally complex data. In Proceedings of SPIN. 123--138.

Digital Library

[21]

Sarfraz Khurshid, Corina S. Păsăreanu, and Willem Visser. 2003. Generalized symbolic execution for model checking and testing. In Proceedings of TACAS. 553--568.

Digital Library

[22]

Sarfraz Khurshid and Yuk Lai Suen. 2005. Generalizing symbolic execution to library classes. In Proceedings of PASTE. 103--110.

Digital Library

[23]

James C. King. 1976. Symbolic execution and program testing. Commun. ACM 19, 7, 385--394.

Digital Library

[24]

Kin-Keung Ma, Khoo Yit Phang, Jeffrey S. Foster, and Michael Hicks. 2011. Directed symbolic execution. In Proceedings of SAS. 95--111.

Digital Library

[25]

Yu-Seung Ma, Jeff Offutt, and Yong Rae Kwon. 2005. MuJava: An automated class mutation system. Softw. Test. Verif. Reliab. 15, 2 (2005), 97--133.

Digital Library

[26]

Eric Mercer, Suzette Person, and Neha Rungta. 2012. Computing and visualizing the impact of change with Java PathFinder extensions. SIGSOFT Softw. Eng. Notes 37, 6 (2012), 1--5.

Digital Library

[27]

Corina S. Păsăreanu, Peter C. Mehlitz, David H. Bushnell, Karen Gundy-Burlet, Michael Lowry, Suzette Person, and Mark Pape. 2008. Combining unit-level symbolic execution and system-level concrete execution for testing NASA software. In Proceedings of ISSTA. 15--25.

Digital Library

[28]

Corina S. Păsăreanu and Neha Rungta. 2010. Symbolic PathFinder: Symbolic execution of Java bytecode. In Proceedings of ASE. 179--180.

Digital Library

[29]

Corina S. Păsăreanu, Neha Rungta, and Willem Visser. 2011. Symbolic execution with mixed concrete-symbolic solving. In Proceedings of ISSTA. 34--44.

Digital Library

[30]

Suzette Person, Matthew B. Dwyer, Sebastian Elbaum, and Corina S. Păsăreanu. 2008. Differential symbolic execution. In Proceedings of FSE. 226--237.

Digital Library

[31]

Suzette Person, Guowei Yang, Neha Rungta, and Sarfraz Khurshid. 2011. Directed incremental symbolic execution. In Proceedings of PLDI. 504--515.

Digital Library

[32]

Dawei Qi, Hoang D. T. Nguyen, and Abhik Roychoudhury. 2011. Path exploration based on symbolic output. In Proceedings of ESEC/FSE. 278--288.

Digital Library

[33]

Dawei Qi, Abhik Roychoudhury, and Zhenkai Liang. 2010. Test generation to expose changes in evolving programs. In Proceedings of ASE. 397--406.

Digital Library

[34]

David A. Ramos and Dawson R. Engler. 2011. Practical, low-effort equivalence verification of real code. In Proceedings of CAV. 669--685.

Digital Library

[35]

Neha Rungta, Suzette Person, and Joshua Branchaud. 2012. A change impact analysis to characterize evolving program behaviors. In Proceedings of ICSM. 109--118.

Digital Library

[36]

SAE-ARP4761. 1996. Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment. SAE International.

[37]

Raul Santelices and Mary Jean Harrold. 2010. Exploiting program dependencies for scalable multiple-path symbolic execution. In Proceedings of ISSTA. 195--206.

Digital Library

[38]

Koushik Sen, Darko Marinov, and Gul Agha. 2005. CUTE: A concolic unit testing engine for C. In Proceedings of ESEC/FSE. 263--272.

Digital Library

[39]

Chiyoung Seo, Sam Malek, and Nenad Medvidovic. 2006. An energy consumption framework for distributed Java-based software systems. Tech. Rep. USC-CSE-2006-604. University of Southern California.

[40]

Junaid Haroon Siddiqui and Sarfraz Khurshid. 2010. ParSym: Parallel symbolic execution. In Proceedings of ICSTE. V1--405--V1--409.

[41]

Junaid Haroon Siddiqui and Sarfraz Khurshid. 2012. Scaling symbolic execution using ranged analysis. In Proceedings of OOPSLA. 523--536.

Digital Library

[42]

Matheus Souza, Mateus Borges, Marcelo d'Amorim, and Corina S. Păsăreanu. 2011. CORAL: Solving complex constraints for symbolic PathFinder. In NASA Formal Methods. Lecture Notes in Computer Science, vol. 6617, Springer, 359--374.

Digital Library

[43]

Matt Staats and Corina S. Păsăreanu. 2010. Parallel symbolic execution for structural test generation. In Proceedings of ISSTA. 183--194.

Digital Library

[44]

Janos Sztipanovits and Gabor Karsai. 2002. Generative programming for embedded systems. In Proceedings of GPCE. 32--49.

Digital Library

[45]

Kunal Taneja, Tao Xie, Nikolai Tillmann, and Jonathan de Halleux. 2011. eXpress: Guided path exploration for efficient regression test generation. In Proceedings of ISSTA. 1--11.

Digital Library

[46]

Willem Visser, Jaco Geldenhuys, and Matthew B. Dwyer. 2012. Green: Reducing, reusing and recycling constraints in program analysis. In Proceedings of FSE. 58:1--58:11.

Digital Library

[47]

Willem Visser, Klaus Havelund, Guillaume P. Brat, Seungjoon Park, and Flavio Lerda. 2003. Model checking programs. Automat. Softw. Eng. 10, 2 (2003), 203--232.

Digital Library

[48]

Zhihong Xu and Gregg Rothermel. 2009. Directed test suite augmentation. In Proceedings of APSEC. 406--413.

Digital Library

[49]

Guowei Yang, Matthew B. Dwyer, and Gregg Rothermel. 2009. Regression model checking. In Proceedings of ICSM. 115--124.

[50]

Guowei Yang, Sarfraz Khurshid, Suzette Person, and Neha Rungta. 2014. Property differencing for incremental checking. In Proceedings of ICSE. to appear.

Digital Library

[51]

Guowei Yang, Corina S. Păsăreanu, and Sarfraz Khurshid. 2012. Memoized symbolic execution. In Proceedings of ISSTA. 144--154.

Digital Library

Cited By

Yi QYu YYang G(2024)Compatible Branch Coverage Driven Symbolic Execution for Efficient Bug FindingProceedings of the ACM on Programming Languages10.1145/36564438:PLDI(1633-1655)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656443
Sun XGupta RBond MLee JPayer H(2024)SSRD: Shapes and Summaries for Race Detection in Concurrent Data StructuresProceedings of the 2024 ACM SIGPLAN International Symposium on Memory Management10.1145/3652024.3665505(68-81)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3652024.3665505
Glock JPichler JPinzger M(2024)PASDA: A partition-based semantic differencing approach with best effort classification of undecided casesJournal of Systems and Software10.1016/j.jss.2024.112037213(112037)Online publication date: Jul-2024
https://doi.org/10.1016/j.jss.2024.112037
Show More Cited By

Index Terms

Directed Incremental Symbolic Execution

Recommendations

Differential symbolic execution
SIGSOFT '08/FSE-16: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering

Detecting and characterizing the effects of software changes is a fundamental component of software maintenance. Version differencing information can be used to perform version merging, infer change characteristics, produce program documentation, and ...
Directed incremental symbolic execution
PLDI '11

The last few years have seen a resurgence of interest in the use of symbolic execution -- a program analysis technique developed more than three decades ago to analyze program execution paths. Scaling symbolic execution and other path-sensitive analysis ...
Directed incremental symbolic execution
PLDI '11: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation

The last few years have seen a resurgence of interest in the use of symbolic execution -- a program analysis technique developed more than three decades ago to analyze program execution paths. Scaling symbolic execution and other path-sensitive analysis ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Software Engineering and Methodology

ACM Transactions on Software Engineering and Methodology Volume 24, Issue 1

September 2014

226 pages

ISSN:1049-331X

EISSN:1557-7392

DOI:10.1145/2676679

Editor:
David S. Rosenblum
National University of Singapore, Singapore

Issue’s Table of Contents

Copyright © 2014 Public Domain.

This paper is authored by an employee(s) of the United States Government and is in the public domain. Non-exclusive copying or redistribution is allowed, provided that the article citation is given and the authors and agency are clearly identified as its source.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2014

Accepted: 01 April 2014

Revised: 01 March 2014

Received: 01 September 2013

Published in TOSEM Volume 24, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
665
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)3

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yi QYu YYang G(2024)Compatible Branch Coverage Driven Symbolic Execution for Efficient Bug FindingProceedings of the ACM on Programming Languages10.1145/36564438:PLDI(1633-1655)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656443
Sun XGupta RBond MLee JPayer H(2024)SSRD: Shapes and Summaries for Race Detection in Concurrent Data StructuresProceedings of the 2024 ACM SIGPLAN International Symposium on Memory Management10.1145/3652024.3665505(68-81)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3652024.3665505
Glock JPichler JPinzger M(2024)PASDA: A partition-based semantic differencing approach with best effort classification of undecided casesJournal of Systems and Software10.1016/j.jss.2024.112037213(112037)Online publication date: Jul-2024
https://doi.org/10.1016/j.jss.2024.112037
Wen JMahmud TChe MYan YYang G(2023)Intelligent Constraint Classification for Symbolic Execution2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00023(144-154)Online publication date: Mar-2023
https://doi.org/10.1109/SANER56733.2023.00023
Yu JWu YXie XLe WMa LChen YHu JZhang FGrundy JPollock LPenta M(2023)GameRTS: A Regression Testing Framework for Video GamesProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00122(1393-1404)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00122
Shahbaz M(2023)An Approach for Test Impact Analysis on the Integration Level in Java ProgramsProceedings of Eighth International Congress on Information and Communication Technology10.1007/978-981-99-3091-3_14(171-188)Online publication date: 30-Jul-2023
https://doi.org/10.1007/978-981-99-3091-3_14
Li SChen CHuang ZDing Z(2023)Change‐aware model checking for evolving concurrent programs based on Program Dependence NetJournal of Software: Evolution and Process10.1002/smr.2626Online publication date: 9-Nov-2023
https://doi.org/10.1002/smr.2626
Nguyen HGrunske LDwyer MDamian DZeller A(2022)BeDivFuzzProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510182(249-261)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510182
Huang HGuo YShi QYao PWu RZhang C(2022)BEACON: Directed Grey-Box Fuzzing with Provable Path Pruning2022 IEEE Symposium on Security and Privacy (SP)10.1109/SP46214.2022.9833751(36-50)Online publication date: May-2022
https://doi.org/10.1109/SP46214.2022.9833751
Yi QYang G(2022)Feedback-Driven Incremental Symbolic Execution2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE55969.2022.00055(505-516)Online publication date: Oct-2022
https://doi.org/10.1109/ISSRE55969.2022.00055
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents