Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
article

An empirical investigation into path divergences for concolic execution using CREST

Published: 01 December 2015 Publication History

Abstract

Recently, concolic execution has become a hotspot in the domain of software testing and program analysis. However, a practical challenge, called path divergence, impairs the soundness and completeness of concolic execution. A path divergence indicates the tested program runs an unpredicted path. In this work, we carry out a comprehensive empirical study on path divergences using an open-source concolic execution tool, named CREST. To make the investigation representative, we select 120 test units randomly from 21 different open-source programs. The results are interesting, and will provide insight to solve the challenging path-divergence problem. First, about one-half of test units suffer from path divergences, indicating path divergences are so prevalent that the issue is worthy of great attention. Second, quite a number of generated test inputs drive test units to take divergent paths. This means testers need considerable effort to eliminate the misleading test inputs before aggregating them to a test suite. Third, we dig out ten divergent patterns through manual analysis of each path divergence. Among them, the three most prevalent ones, which are exceptions, external calls, and type casts, lead to almost 82% of path divergences. Finally, we discuss several countermeasures to overcome path divergences. Copyright © 2015 John Wiley & Sons, Ltd.

References

[1]
King JC. Symbolic execution and program testing. Journal of the ACM 1976; Volume 19 Issue 7: pp.385-394.
[2]
Myers GJ. The Art of Software Testing. Wiley: New York, USA, 1979.
[3]
Burnim J, Sen K. Heuristics for scalable dynamic test generation. In Proceedings of 23rd IEEE/ACM International Conference on Automated Software Engineering, 443-446, 2008.
[4]
Chen T, Zhang XS, Guo SZ, Li HY, Wu Y. State of the art: dynamic symbolic execution for automated test generation. Future Generation Computer Systems 2013; Volume 29 Issue 7: pp.1758-1773.
[5]
Godefroid P, Klarlund N, Sen K. DART: directed automated random testing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation PLDI, 2013213-223, 2005.
[6]
Păsăreanu CS, Rungta N, Visser W. Symbolic execution with mixed concrete-symbolic solving. In Proceedings of the International Symposium on Software Testing and Analysis, 34-44, 2011.
[7]
Godefroid P, Levin M, Molnar D. Automated whitebox fuzz testing. In Proceedings of the Network and Distributed System Security Symposium. 2008
[8]
Anand S, Naik M, Harrold MJ, Yang H. Automated concolic testing of smartphone apps. In Proceedings of the 20th ACM SIGSOFT International Symposium on the Foundations of Software Engineering, 59-73. 2012
[9]
Chen T, Zhang XS, Zhu C, Ji XL, Guo SZ, Wu Y. Design and implementation of a dynamic symbolic execution tool for windows executables. Journal of Software: Evolution and Process 2013; Volume 25 Issue 12: pp.1249-1272.
[10]
Christakis M, Patrice G. Proving memory safety of the ANI windows image parser using compositional exhaustive testing. Technical Report, MSR-TR-2013-120, Microsoft Research. 2013
[11]
Ma KK, Phang KY, Foster JS, Hicks M. Directed symbolic execution. In Proceedings of 18th International Symposium on Static Analysis, 95-111. 2011
[12]
Davies M, Păsăreanu CS, Raman V. Symbolic execution enhanced system testing. Verified software: theories, tools, experiments, 294-309. 2012
[13]
Sapra S, Minea M, Chaki S, Gurfinkel A, Clarke EM. Finding errors in Python programs using dynamic symbolic execution. Testing Software and Systems, 283-289. 2013
[14]
Necula GC, McPeak S, Rahul SP, Weimer W. CIL: intermediate language and tools for analysis and transformation of C programs. In Proceedings of Conference on compiler Construction, 213-228. 2002
[15]
CREST: automated test generation tool for C. "http://jburnim.github.io/crest/".
[16]
Qu X, Robinson B. A case study of concolic testing tools and their limitations. In Proceedings of the International Symposium on Empirical Software Engineering and Measurement, 117-126. 2011
[17]
Joshi P, Sen K, Shlimovich M. Predictive testing: amplifying the effectiveness of software testing. In Proceeding of the 6th Joint Meeting on European software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering, 561-564. 2007
[18]
Lakhotia K, McMinn P, Harman M. An empirical investigation into branch coverage for C programs using CUTE and AUSTIN. Journal of Systems and Software 2010; Volume 83 Issue 12: pp.2379-2391.
[19]
8conv: converts quoted-printable, UTF-8, UTF-16BE, UTF16LE to 8-bit. "http://eightconv.sourceforge.net/."
[20]
inih: Simple .INI file parser in C, good for embedded systems. "https://code.google.com/p/inih/".
[21]
Basic Compression Library. "http://bcl.comli.eu/".
[22]
Yices: An SMT Solver. "http://yices.csl.sri.com/."
[23]
FLAC: Free lossless audio codec. "http://xiph.org/flac/."
[24]
libhdate, hcal and hdate: C library for Hebrew dates / times of day / solar times. "http://libhdate.sourceforge.net/."
[25]
Chipounov V, Kuznetsov V, Candea G. S2E: a platform for in-vivo multi-path analysis of software systems. Computer Architecture News 2011; Volume 39 Issue 1: pp.265-278.
[26]
Godefroid P. Compositional dynamic test generation. In Proceedings of the 34th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, 47-54. 2007
[27]
Schwartz EJ, Avgerinos T, Brumley D. All you ever wanted to know about dynamic taint analysis and forward symbolic execution but might have been afraid to ask. In Proceedings of IEEE Symposium on Security and Privacy, 317-331. 2010
[28]
Godefroid P, Lahiri SK, Gonz¿alez CR. Incremental compositional dynamic test generation. Technique report MSR-TR-2010-11. 2010
[29]
Cadar C, Dunbar D, Engler D. Klee: unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the USENIX Symposium on Operating System Design and Implementation. 2008
[30]
Elkarablieh B, Godefroid P, Levin MY. Precise pointer reasoning for dynamic test generation. In Proceedings of International Conference on Software Testing and Analysis, 129-140. 2009
[31]
Kosmatov N. All-paths test generation for programs with internal aliases. In Proceedings of IEEE International Symposium on Software Reliability Engineering, 147-156. 2008
[32]
Papadakis M, Malevris N. Mutation based test case generation via a path exploration strategy. Information and Software Technology 2012; Volume 54 Issue 9: pp.915-932.
[33]
Collingbourne P, Cadar C, Kelly PHJ. Symbolic crosschecking of floating-point and SIMD code. In Proceedings of the EuroSys, 315-328. 2011
[34]
Leonardo DM, Nikolaj B. Z3: an efficient SMT solver. Lecture Notes in Computer Science 2008; Volume 4963: pp.337-340.
[35]
Blanc B, Bouquet F, Gotlieb A, Jeannet B, Jeron T, Legeard B, Marre B, Michel C, Rueher M. The V3F project. In Proceedings of the 1st Workshop on Constraints in Software Testing, Verification and Analysis, 2006
[36]
Botella B, Gotlieb A, Michel C. Symbolic execution of floating-point computations. Software Testing, Verification and Reliability 2006; Volume 16 Issue 2: pp.97-121.
[37]
Bagnara R, Carlier M, Gori R, Gotlieb A. Symbolic path-oriented test data generation for floating-point programs. In Proceedings of the 6th IEEE International Conference on Software Testing, Verification and Validation. 2013
[38]
Lakhotia K, Tillmann N, Harman M, <familyNamePrefix>de</familyNamePrefix>Halleux J. FloPSy - search-based floating point constraint solving for symbolic execution. In Proceedings of 22nd IFIP WG6.1 International Conference on Testing Software and Systems, 142-157. 2010
[39]
Arcuri A. Theoretical analysis of local search in software testing. In Proceedings of the 5th International Symposium on Stochastic Algorithms: Foundations and Applications, 156-168. 2009
[40]
Harman M, McMinn P. A theoretical and empirical study of search-based testing: Local, global, and hybrid search ., IEEE Transactions on Software Engineering, Vol. 36, No.2, 226-247. 2010
[41]
Brumley D, Hartwig C, Kang MG, Liang ZK, Newsome J, Poosankam P, Song D. Bitscope: automatically dissecting malicious binaries. Technical Report, CS-07-133. 2007
[42]
Fuzzgrind: an automatic fuzzing tool. Fuzzgrind. "http://seclab.sogeti.com/dotclear/index.php?pages/Fuzzgrind."
[43]
Tillmann N, De Halleux J. Pex-white box test generation for .NET. In Proceedings of 2nd International Conference on Tests and Proofs, 134-153. 2008

Cited By

View all
  • (2024)Marco: A Stochastic Asynchronous Concolic ExplorerProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623301(1-12)Online publication date: 20-May-2024
  • (2018)A Survey of Symbolic Execution TechniquesACM Computing Surveys10.1145/318265751:3(1-39)Online publication date: 23-May-2018

Index Terms

  1. An empirical investigation into path divergences for concolic execution using CREST
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Security and Communication Networks
    Security and Communication Networks  Volume 8, Issue 18
    December 2015
    1151 pages
    ISSN:1939-0114
    EISSN:1939-0122
    Issue’s Table of Contents

    Publisher

    John Wiley & Sons, Inc.

    United States

    Publication History

    Published: 01 December 2015

    Author Tags

    1. concolic execution
    2. countermeasures
    3. divergent patterns
    4. misleading inputs
    5. path divergences
    6. prevalence

    Qualifiers

    • Article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 22 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Marco: A Stochastic Asynchronous Concolic ExplorerProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623301(1-12)Online publication date: 20-May-2024
    • (2018)A Survey of Symbolic Execution TechniquesACM Computing Surveys10.1145/318265751:3(1-39)Online publication date: 23-May-2018

    View Options

    View options

    Get Access

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media