research-article

Steering symbolic execution to less traveled paths

Authors:

Linzhang Wang, and

Xuandong LiAuthors Info & Claims

OOPSLA '13: Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications

October 2013

Pages 19 - 32

https://doi.org/10.1145/2509136.2509553

Published: 29 October 2013 Publication History

Abstract

Symbolic execution is a promising testing and analysis methodology. It systematically explores a program's execution space and can generate test cases with high coverage. One significant practical challenge for symbolic execution is how to effectively explore the enormous number of program paths in real-world programs. Various heuristics have been proposed for guiding symbolic execution, but they are generally inefficient and ad-hoc. In this paper, we introduce a novel, unified strategy to guide symbolic execution to less explored parts of a program. Our key idea is to exploit a specific type of path spectra, namely the length-n subpath program spectra, to systematically approximate full path information for guiding path exploration. In particular, we use frequency distributions of explored length-n subpaths to prioritize "less traveled" parts of the program to improve test coverage and error detection. We have implemented our general strategy in KLEE, a state-of-the-art symbolic execution engine. Evaluation results on the GNU Coreutils programs show that (1) varying the length n captures program-specific information and exhibits different degrees of effectiveness, and (2) our general approach outperforms traditional strategies in both coverage and error detection.

References

[1]

S. Anand, C. S. Păsăreanu, and W. Visser. JPF--SE: A symbolic execution extension to Java PathFinder. In Tools and Algorithms for the Construction and Analysis of Systems, pages 134--138. Springer, 2007.

Digital Library

[2]

V. Bala, E. Duesterwald, and S. Banerjia. Transparent dynamic optimization: The design and implementation of Dynamo. Technical report, Technical Report HPL-1999-78, Hewlett-Packard Laboratories, 1999.

[3]

V. Bala, E. Duesterwald, and S. Banerjia. Dynamo: a transparent dynamic optimization system. In ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 1--12. ACM, 2000.

Digital Library

[4]

T. Ball and J. Larus. Efficient path profiling. In ACM/IEEE International Symposium on Microarchitecture, pages 46--57. IEEE Computer Society, 1996.

Digital Library

[5]

P. Boonstoppel, C. Cadar, and D. Engler. RWset: Attacking path explosion in constraint-based test generation. In International conference on Tools and Algorithms for the Construction and Analysis of Systems, pages 351--366. Springer, 2008.

Digital Library

[6]

J. Burnim and K. Sen. Heuristics for scalable dynamic test generation. In IEEE/ACM International Conference on Automated Software Engineering, pages 443--446. IEEE Computer Society, 2008.

Digital Library

[7]

C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In USENIX Symposium on Operating Systems Design and Implementation, pages 209--224, 2008.

Digital Library

[8]

C. Cadar, V. Ganesh, P. Pawlowski, D. Dill, and D. Engler. EXE: automatically generating inputs of death. ACM Transactions on Information and System Security (TISSEC), 12(2):10, 2008.

Digital Library

[9]

C. Cadar, P. Godefroid, S. Khurshid, C. Pasareanu, K. Sen, N. Tillmann, and W. Visser. Symbolic execution for software testing in practice: preliminary assessment. In International Conference on Software Engineering, pages 1066--1071. IEEE, 2011.

Digital Library

[10]

L. A. Clarke. A system to generate test data and symbolically execute programs. IEEE Transactions on Software Engineering, 2(3):215--222, 1976.

Digital Library

[11]

Coreutils - GNU core utilities. http://www.gnu.org/software/coreutils/.

[12]

R. DeMillo, R. Lipton, and F. Sayward. Hints on test data selection: Help for the practicing programmer. Computer, 11(4):34--41, 1978.

Digital Library

[13]

E. Duesterwald and V. Bala. Software profiling for hot path prediction: Less is more. In International Conference on Architectural Support for Programming Languages and Operating Systems, pages 202--211. ACM, 2000.

Digital Library

[14]

J. W. Duran and S. C. Ntafos. An evaluation of random testing. IEEE Transactions on Software Engineering, 10(4):438--444, 1984.

Digital Library

[15]

R. Ferguson and B. Korel. The chaining approach for software test data generation. ACM Transactions on Software Engineering and Methodology (TOSEM), 5(1):63--86, 1996.

Digital Library

[16]

P. Godefroid. Compositional dynamic test generation. In ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 47--54. ACM, 2007.

Digital Library

[17]

P. Godefroid, N. Klarlund, and K. Sen. DART: directed automated random testing. In ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 213--223. ACM, 2005.

Digital Library

[18]

P. Godefroid, M. Levin, D. Molnar, et al. Automated whitebox fuzz testing. In Network and Distributed System Security Symposium. The Internet Society, 2008.

[19]

D. Hamlet and R. Taylor. Partition testing does not inspire confidence {program testing}. IEEE Transactions on Software Engineering, 16(12):1402--1411, 1990.

Digital Library

[20]

M. Harrold, G. Rothermel, K. Sayre, R. Wu, and L. Yi. An empirical investigation of the relationship between spectra differences and regression faults. Software Testing Verification and Reliability, 10(3):171--194, 2000.

[21]

Y. Jia and M. Harman. Milu: A customizable, runtime-optimized higher order mutation testing tool for the full C language. In Testing: Academia and Industry Conference - Practice And Research Techniques, pages 94--98. IEEE, 2008.

Digital Library

[22]

J. King. Symbolic execution and program testing. Communications of the ACM, 19(7):385--394, 1976.

Digital Library

[23]

The KLEE symbolic virtual machine. http://klee.llvm.org/.

[24]

R. Majumdar and K. Sen. Hybrid concolic testing. In International Conference on Software Engineering, pages 416--426. IEEE, 2007.

Digital Library

[25]

D. Marinov, A. Andoni, D. Daniliuc, S. Khurshid, and M. Rinard. An evaluation of exhaustive testing for data structures. Technical report, Technical Report MIT-LCS-TR-921, MIT CSAIL, Cambridge, MA, 2003.

[26]

C. S. Păsăreanu and W. Visser. Verification of Java programs using symbolic execution and invariant generation. In Model Checking Software, pages 164--181. Springer, 2004.

[27]

T. Reps, T. Ball, M. Das, and J. Larus. The use of program profiling for software maintenance with applications to the Year 2000 problem. In Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 432--449, 1997.

Digital Library

[28]

M. Staats and C. Păsăreanu. Parallel symbolic execution for structural test generation. In International Symposium on Software Testing and Analysis, pages 183--194. ACM, 2010.

Digital Library

[29]

K. Taneja, T. Xie, N. Tillmann, and J. de Halleux. eXpress: guided path exploration for efficient regression test generation. In International Symposium on Software Testing and Analysis, pages 1--11. ACM, 2011.

Digital Library

[30]

N. Tillmann and J. De Halleux. Pex - white box test generation for .NET. Tests and Proofs, pages 134--153, 2008.

Digital Library

[31]

W. Visser, C. S. Păsăreanu, and R. Pelánek. Test input generation for Java containers using state matching. In International Symposium on Software Testing and Analysis, pages 37--48. ACM, 2006.

Digital Library

[32]

T. Xie, N. Tillmann, J. de Halleux, and W. Schulte. Fitness-guided path exploration in dynamic symbolic execution. In IEEE/IFIP International Conference on Dependable Systems and Networks, pages 359--368. IEEE, 2009.

[33]

X. Yang, Y. Chen, E. Eide, and J. Regehr. Finding and understanding bugs in C compilers. In ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 283--294. ACM, 2012.

Digital Library

Cited By

Yoon JCha S(2024)FeatMaker: Automated Feature Engineering for Search Strategy of Symbolic ExecutionProceedings of the ACM on Software Engineering10.1145/36608151:FSE(2447-2468)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660815
Sun YYang GLv SLi ZSun LRoychoudhury APaiva AAbreu RStorey M(2024)Concrete Constraint Guided Symbolic ExecutionProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639078(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639078
Shi JXiao YLi YLi YYu DYu CSu HChen YHuo WJust RFraser G(2023)ACETest: Automated Constraint Extraction for Testing Deep Learning OperatorsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598088(690-702)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598088
Show More Cited By

Recommendations

Steering symbolic execution to less traveled paths
OOPSLA '13

Symbolic execution is a promising testing and analysis methodology. It systematically explores a program's execution space and can generate test cases with high coverage. One significant practical challenge for symbolic execution is how to effectively ...
Read More
Learning to Explore Paths for Symbolic Execution
CCS '21: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security

Symbolic execution is a powerful technique that can generate tests steering program execution into desired paths. However, the scalability of symbolic execution is often limited by path explosion, i.e., the number of symbolic states representing the ...
Read More
Scaling symbolic execution using staged analysis

Recent advances in constraint solving technology and raw computation power have led to a substantial increase in the effectiveness of techniques based on symbolic execution for systematic bug finding. However, scaling symbolic execution remains a ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

OOPSLA '13: Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications

October 2013

904 pages

ISBN:9781450323741

DOI:10.1145/2509136

Co-chair:
Antony Hosking
Purdue University, USA
,
General Chair:
Patrick Eugster
Purdue University, USA
,
Program Chair:
Cristina V. Lopes
University of California, Irvine, USA

ACM SIGPLAN Notices Volume 48, Issue 10
OOPSLA '13
October 2013
867 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/2544173
Editor:
Mark W. Bailey
Hamilton College, Clinton, NY
Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SPLASH '13

Sponsor:

SIGPLAN

SPLASH '13: Conference on Systems, Programming, and Applications: Software for Humanity

October 29 - 31, 2013

Indiana, Indianapolis, USA

Acceptance Rates

OOPSLA '13 Paper Acceptance Rate 50 of 189 submissions, 26%;

Overall Acceptance Rate 268 of 1,244 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

85
Total Citations
View Citations
856
Total Downloads

Downloads (Last 12 months)88
Downloads (Last 6 weeks)8

Other Metrics

View Author Metrics

Citations

Cited By

Yoon JCha S(2024)FeatMaker: Automated Feature Engineering for Search Strategy of Symbolic ExecutionProceedings of the ACM on Software Engineering10.1145/36608151:FSE(2447-2468)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660815
Sun YYang GLv SLi ZSun LRoychoudhury APaiva AAbreu RStorey M(2024)Concrete Constraint Guided Symbolic ExecutionProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639078(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639078
Shi JXiao YLi YLi YYu DYu CSu HChen YHuo WJust RFraser G(2023)ACETest: Automated Constraint Extraction for Testing Deep Learning OperatorsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598088(690-702)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598088
Cha SLee MLee SOh HDwyer MDamian DZeller A(2022)SymTunerProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510185(2068-2079)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510185
Cha SHong SBak JKim JLee JOh H(2022)Enhancing Dynamic Symbolic Execution by Automatically Learning Search HeuristicsIEEE Transactions on Software Engineering10.1109/TSE.2021.310187048:9(3640-3663)Online publication date: 1-Sep-2022
https://doi.org/10.1109/TSE.2021.3101870
Soremekun EPavese EHavrikov NGrunske LZeller A(2022)Inputs From Hell:IEEE Transactions on Software Engineering10.1109/TSE.2020.301371648:4(1138-1153)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TSE.2020.3013716
Hu YShen ZDolan-Gavitt B(2022)Characterizing and Improving Bug-Finders with Synthetic Bugs2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER53432.2022.00115(971-982)Online publication date: Mar-2022
https://doi.org/10.1109/SANER53432.2022.00115
Yi QYang G(2022)Feedback-Driven Incremental Symbolic Execution2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE55969.2022.00055(505-516)Online publication date: Oct-2022
https://doi.org/10.1109/ISSRE55969.2022.00055
Huang ZWhite M(2022)Semantic-Aware Vulnerability Detection2022 IEEE International Conference on Cyber Security and Resilience (CSR)10.1109/CSR54599.2022.9850330(68-75)Online publication date: 27-Jul-2022
https://doi.org/10.1109/CSR54599.2022.9850330
Situ LWang LLiu YMao BLi X(2022)Automatic Detection and Repair Recommendation for Missing ChecksJournal of Computer Science and Technology10.1007/s11390-019-1955-334:5(972-992)Online publication date: 11-Mar-2022
https://dl.acm.org/doi/10.1007/s11390-019-1955-3
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents