research-article

Parallel symbolic execution for automated real-world software testing

Authors:

Cristian Zamfir,

George CandeaAuthors Info & Claims

EuroSys '11: Proceedings of the sixth conference on Computer systems

Pages 183 - 198

https://doi.org/10.1145/1966445.1966463

Published: 10 April 2011 Publication History

Abstract

This paper introduces Cloud9, a platform for automated testing of real-world software. Our main contribution is the scalable parallelization of symbolic execution on clusters of commodity hardware, to help cope with path explosion. Cloud9 provides a systematic interface for writing "symbolic tests" that concisely specify entire families of inputs and behaviors to be tested, thus improving testing productivity. Cloud9 can handle not only single-threaded programs but also multi-threaded and distributed systems. It includes a new symbolic environment model that is the first to support all major aspects of the POSIX interface, such as processes, threads, synchronization, networking, IPC, and file I/O. We show that Cloud9 can automatically test real systems, like memcached, Apache httpd, lighttpd, the Python interpreter, rsync, and curl. We show how Cloud9 can use existing test suites to generate new test cases that capture untested corner cases (e.g., network stream fragmentation). Cloud9 can also diagnose incomplete bug fixes by analyzing the difference between buggy paths before and after a patch.

References

[1]

Amazon. Amazon EC2. http://aws.amazon.com/ec2.

[2]

Jiri Barnat, Lubos Brim, and Petr Rockai. Scalable multi-core LTL model-checking. In Intl. SPIN Workshop, 2007.

Digital Library

[3]

Peter Boonstoppel, Cristian Cadar, and Dawson R. Engler. RWset: Attacking path explosion in constraint-based test generation. In Intl. Conf. on Tools and Algorithms for the Construction and Analysis of Systems, 2008.

Digital Library

[4]

Cristian Cadar, Daniel Dunbar, and Dawson R. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Symp. on Operating Systems Design and Implementation, 2008.

Digital Library

[5]

Cristian Cadar, Vijay Ganesh, Peter M. Pawlowski, David L. Dill, and Dawson R. Engler. EXE: Automatically generating inputs of death. In Conf. on Computer and Communication Security, 2006.

Digital Library

[6]

Vitaly Chipounov, Vlad Georgescu, Cristian Zamfir, and George Candea. Selective symbolic execution. In Workshop on Hot Topics in Dependable Systems, 2009.

[7]

Vitaly Chipounov, Volodymyr Kuznetsov, and George Candea. S2E: A platform for in-vivo multi-path analysis of software systems. In Intl. Conf. on Architectural Support for Programming Languages and Operating Systems, 2011.

Digital Library

[8]

Liviu Ciortea, Cristian Zamfir, Stefan Bucur, Vitaly Chipounov, and George Candea. Cloud9: A software testing service. In Workshop on Large Scale Distributed Systems and Middleware, 2009.

[9]

Coreutils. Coreutils. http://www.gnu.org/software/coreutils/.

[10]

Eucalyptus. Eucalyptus software. http://open.eucalyptus.com/, 2010.

[11]

P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In Conf. on Programming Language Design and Implementation, 2005.

Digital Library

[12]

Patrice Godefroid. Model checking for programming languages using VeriSoft. In Symp. on Principles of Programming Languages, 1997.

Digital Library

[13]

Patrice Godefroid. Compositional dynamic test generation. In Symp. on Principles of Programming Languages, 2007. Extended abstract.

Digital Library

[14]

Patrice Godefroid, Michael Y. Levin, and David Molnar. Automated whitebox fuzz testing. In Network and Distributed System Security Symp., 2008.

[15]

Orna Grumberg, Tamir Heyman, and Assaf Schuster. A work-efficient distributed algorithm for reachability analysis. Formal Methods in System Design, 29 (2), 2006.

Digital Library

[16]

G. J. Holzmann and D. Bosnacki. Multi-core model checking with SPIN. In Intl. Parallel and Distributed Processing Symp., 2007.

[17]

Gerard J. Holzmann, Rajeev Joshi, and Alex Groce. Tackling large verification problems with the Swarm tool. In Intl. SPIN Workshop, 2008.

Digital Library

[18]

Rahul Kumar and Eric G. Mercer. Load balancing parallel explicit state model checking. In Intl. Workshop on Parallel and Distributed Methods in Verification, 2004.

[19]

Flavio Lerda and Riccardo Sisto. Distributed-memory model checking with SPIN. In Intl. SPIN Workshop, 1999.

Digital Library

[20]

Rupak Majumdar and Koushik Sen. Hybrid concolic testing. In Intl. Conf. on Software Engineering, 2007.

Digital Library

[21]

Paul D. Marinescu and George Candea. LFI: A practical and general library-level fault injector. In Intl. Conf. on Dependable Systems and Networks, 2009.

[22]

Steve McConnell. Code Complete. Microsoft Press, 2004.

[23]

Sasa Misailovic, Aleksandar Milicevic, Nemanja Petrovic, Sarfraz Khurshid, and Darko Marinov. Parallel test generation and execution with Korat. In Symp. on the Foundations of Software Eng., 2007.

Digital Library

[24]

Madanlal Musuvathi, Shaz Qadeer, Thomas Ball, Gérard Basler, Piramanayagam Arumuga Nainar, and Iulian Neamtiu. Finding and reproducing Heisenbugs in concurrent programs. In Symp. on Operating Systems Design and Implementation, 2008.

Digital Library

[25]

RedHat. RedHat security. http://www.redhat.com/security/updates/classification, 2005.

[26]

Matt Staats and Corina Păsăreanu. Parallel symbolic execution for structural test generation. In Intl. Symp. on Software Testing and Analysis, 2010.

Digital Library

[27]

Ulrich Stern and David L. Dill. Parallelizing the Murφ verifier. In Intl. Conf. on Computer Aided Verification, 1997.

Digital Library

Cited By

Wilson JAsplund M(2025)Analysing TLS Implementations Using Full-Message Symbolic ExecutionSecure IT Systems10.1007/978-3-031-79007-2_15(283-302)Online publication date: 29-Jan-2025
https://doi.org/10.1007/978-3-031-79007-2_15
Wang HTang ZTan SWang JLiu YFang HXia CWang ZRoychoudhury APaiva AAbreu RStorey M(2024)Combining Structured Static Code Information and Dynamic Symbolic Traces for Software Vulnerability PredictionProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639212(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639212
Coutinho JLemos ATerra-Neves MRibeiro AManquinho VQuintino RMatejczyk B(2024)BugOut: Automated Test Generation and Bug Detection for Low-Code2024 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST60714.2024.00041(373-382)Online publication date: 27-May-2024
https://doi.org/10.1109/ICST60714.2024.00041
Show More Cited By

Index Terms

Parallel symbolic execution for automated real-world software testing

Recommendations

Enhancing symbolic execution with veritesting
ICSE 2014: Proceedings of the 36th International Conference on Software Engineering

We present MergePoint, a new binary-only symbolic execution system for large-scale and fully unassisted testing of commodity off-the-shelf (COTS) software. MergePoint introduces veritesting, a new technique that employs static symbolic execution to ...
Combined Symbolic and Concrete Execution of TTCN-3 for Automated Testing
ISISE '08: Proceedings of the 2008 International Symposium on Information Science and Engieering - Volume 01

Testing procedure can be described by the Testing and Test Control Notation-version 3(TTCN-3). The automatic execution of TTCN-3 test scripts is transformed to automatic testing of system under test (SUT). We propose a framework, which uses combined ...
Concolic testing
ASE '07: Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering

Concolic testing automates test input generation by combining the concrete and symbolic (concolic) execution of the code under test. Traditional test input generation techniques use either (1) concrete execution or (2) symbolic execution that builds ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EuroSys '11: Proceedings of the sixth conference on Computer systems

April 2011

370 pages

ISBN:9781450306348

DOI:10.1145/1966445

General Chair:
Christoph Kirsch
University of Salzburg, Austria
,
Program Chair:
Gernot Heiser
University of New South Wales, Australia

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 April 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

EuroSys '11

Sponsor:

SIGOPS

EuroSys '11: Sixth EuroSys Conference 2011

April 10 - 13, 2011

Salzburg, Austria

Acceptance Rates

EuroSys '11 Paper Acceptance Rate 24 of 161 submissions, 15%;

Overall Acceptance Rate 241 of 1,308 submissions, 18%

Upcoming Conference

EuroSys '25

Sponsor:
sigops

Twentieth European Conference on Computer Systems

March 30 - April 3, 2025

Rotterdam , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

187
Total Citations
View Citations
1,342
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)14

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wilson JAsplund M(2025)Analysing TLS Implementations Using Full-Message Symbolic ExecutionSecure IT Systems10.1007/978-3-031-79007-2_15(283-302)Online publication date: 29-Jan-2025
https://doi.org/10.1007/978-3-031-79007-2_15
Wang HTang ZTan SWang JLiu YFang HXia CWang ZRoychoudhury APaiva AAbreu RStorey M(2024)Combining Structured Static Code Information and Dynamic Symbolic Traces for Software Vulnerability PredictionProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639212(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639212
Coutinho JLemos ATerra-Neves MRibeiro AManquinho VQuintino RMatejczyk B(2024)BugOut: Automated Test Generation and Bug Detection for Low-Code2024 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST60714.2024.00041(373-382)Online publication date: 27-May-2024
https://doi.org/10.1109/ICST60714.2024.00041
Kim BGangadharan D(2024)Edge-Server Workload Characterization in Vehicular Computation Offloading: Semantics and Empirical AnalysisIEEE Access10.1109/ACCESS.2024.341915612(89082-89097)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3419156
Haltermann JJakobs MRichter CWehrheim H(2024)Parallel program analysis on path rangesScience of Computer Programming10.1016/j.scico.2024.103154238:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.scico.2024.103154
Singh ASarita YMendis CSingh G(2024)ConstraintFlow: A Declarative DSL for Easy Development of DNN CertifiersStatic Analysis10.1007/978-3-031-74776-2_16(407-424)Online publication date: 20-Oct-2024
https://dl.acm.org/doi/10.1007/978-3-031-74776-2_16
Wen JMahmud TChe MYan YYang G(2023)Intelligent Constraint Classification for Symbolic Execution2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00023(144-154)Online publication date: Mar-2023
https://doi.org/10.1109/SANER56733.2023.00023
Wei GJia SGao RDeng HTan SBračevac ORompf T(2023)Compiling Parallel Symbolic Execution with Continuations2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00116(1316-1328)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00116
Paduraru CCernat MStaicu A(2023)Concolic execution for RPA testing2023 27th International Conference on Engineering of Complex Computer Systems (ICECCS)10.1109/ICECCS59891.2023.00031(187-196)Online publication date: 14-Jun-2023
https://doi.org/10.1109/ICECCS59891.2023.00031
Kim BKang E(2023)Toward Large-Scale Test for Certifying Autonomous Driving Software in Collaborative Virtual EnvironmentIEEE Access10.1109/ACCESS.2023.329550011(72641-72654)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3295500
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten