Correctness of software has become increasingly important and difficult as programs become more complicated and have more impact on our day-to-day lives. There are two approaches to ensure the correctness of software. Testing is the approach widely used in industry. Today, testing is tedious, expensive and prone to leave errors undetected. The other approach is to verify the correctness or guarantee the proper behavior of software through static analysis and model checking. However, this approach does not scale well, are restricted to simple properties or overwhelm the user with many false alarms. In the recent years, testing and verification have come closer together. Directed testing or concolic testing generates tests from constraints generated through both symbolic and real executions. However, the basic concolic execution algorithms do not scale to larger programs and cannot identify or seek out many types of bugs. This dissertation extends the basic concolic execution algorithm to scale to larger programs and more complex properties.
Specifically, this dissertation presents four symbolic execution algorithms that automatically and systematically generate tests. These algorithms reduce the input space of automated testing and find different classes of errors. Symbolic grammars are introduced to generate orders of magnitude less input strings without sacrificing coverage. Symmetry reduces redundant tests by showing that some parts of the input are independent from other parts. Ideas in Liveness allow test generation to find errors leading to non-termination. Abstraction allows larger inputs to be generated that lead to memory safety violations and thus stop security holes before they happen.
This work has resulted in a tool that generates tests for C programs called S PLAT . S PLAT was used on a wide variety of open-source programs that compare these techniques to conventional industry-wide practices and state-of-the-art research. Preliminary studies show that these ideas are effective in finding new bugs quicker and can explore more of the program than other approaches.
Cited By
- Zhao B and Liu P Behavior Decomposition Proceedings of the 16th International Symposium on Research in Attacks, Intrusions, and Defenses - Volume 8145, (244-264)
- Systematic testing for control applications Proceedings of the Eighth ACM/IEEE International Conference on Formal Methods and Models for Codesign, (1-10)
Recommendations
Test generation via Dynamic Symbolic Execution for mutation testing
ICSM '10: Proceedings of the 2010 IEEE International Conference on Software MaintenanceMutation testing has been used to assess and improve the quality of test inputs. Generating test inputs to achieve high mutant-killing ratios is important in mutation testing. However, existing test-generation techniques do not provide effective support ...
Parallel test generation and execution with Korat
ESEC-FSE '07: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineeringWe present novel algorithms for parallel testing of code that takes structurally complex test inputs. The algorithms build on the Korat algorithm for constraint-based generation of structurally complex test inputs. Given an imperative predicate that ...
Model-based test case generation using symbolic execution
JAMAICA 2013: Proceedings of the 2013 International Workshop on Joining AcadeMiA and Industry Contributions to testing AutomationIn this paper, we present a test case generation method in which test cases are generated from Excel-based functional specications, called error-check/update specications. This method has the following two characteristics. 1) Logical structures and ...