research-article

Verifying systems rules using rule-directed symbolic execution

Authors:

Junfeng YangAuthors Info & Claims

ACM SIGARCH Computer Architecture News, Volume 41, Issue 1

Pages 329 - 342

https://doi.org/10.1145/2490301.2451152

Published: 16 March 2013 Publication History

Abstract

Systems code must obey many rules, such as "opened files must be closed." One approach to verifying rules is static analysis, but this technique cannot infer precise runtime effects of code, often emitting many false positives. An alternative is symbolic execution, a technique that verifies program paths over all inputs up to a bounded size. However, when applied to verify rules, existing symbolic execution systems often blindly explore many redundant program paths while missing relevant ones that may contain bugs.

Our key insight is that only a small portion of paths are relevant to rules, and the rest (majority) of paths are irrelevant and do not need to be verified. Based on this insight, we create WOODPECKER, a new symbolic execution system for effectively checking rules on systems programs. It provides a set of builtin checkers for common rules, and an interface for users to easily check new rules. It directs symbolic execution toward the program paths relevant to a checked rule, and soundly prunes redundant paths, exponentially speeding up symbolic execution. It is designed to be heuristic-agnostic, enabling users to leverage existing powerful search heuristics.

Evaluation on 136 systems programs totaling 545K lines of code, including some of the most widely used programs, shows that, with a time limit of typically just one hour for each verification run, WOODPECKER effectively verifies 28.7% of the program and rule combinations over bounded input, whereas an existing symbolic execution system KLEE verifies only 8.5%. For the remaining combinations, WOODPECKER verifies 4.6 times as many relevant paths as KLEE. With a longer time limit, WOODPECKER verifies much more paths than KLEE, e.g., 17 times as many with a fourhour limit. WOODPECKER detects 113 rule violations, including 10 serious data loss errors with 2 most serious ones already confirmed by the corresponding developers.

References

[1]

The LLVM compiler framework. http://llvm.org.

[2]

STP Constraint Solver. https://sites.google.com/site/stpfastprover/.

[3]

H. Agrawal and J. R. Horgan. Dynamic program slicing. In Proceedings of the ACM SIGPLAN '90 Conference on Programming Language Design and Implementation (PLDI '90), pages 246--256, 1990.

Digital Library

[4]

G. Altekar and I. Stoica. ODR: output-deterministic replay for multicore debugging. In Proceedings of the 22nd ACM Symposium on Operating Systems Principles (SOSP '09), pages 193--206, Oct. 2009.

Digital Library

[5]

A. Andoni, D. Daniliuc, S. Khurshid, and D. Marinov. Evaluating the "small scope hypothesis". Technical report, MIT CSAIL, 2002.

[6]

D. Avots, M. Dalton, V. B. Livshits, and M. S. Lam. Improving software security with a C pointer analysis. In Proceedings of the 27th International Conference on Software Engineering (ICSE '05), pages 332--341, May 2005.

Digital Library

[7]

T. Ball and S. K. Rajamani. Automatically validating temporal safety properties of interfaces. In Proceedings of the Eighth International SPIN Workshop on Model Checking of Software (SPIN '01), pages 103--122, May 2001.

Digital Library

[8]

BBC. The life of birds documentary.

[9]

S. Bucur, V. Ureche, C. Zamfir, and G. Candea. Parallel symbolic execution for automated real-world software testing. In Proceedings of the 6th ACM European Conference on Computer Systems (EUROSYS '11), pages 183--198, 2011.

Digital Library

[10]

C. Cadar. Private email communication, Mar. 2012.

[11]

C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. EXE: automatically generating inputs of death. In Proceedings of the 13th ACM conference on Computer and communications security (CCS '06), pages 322--335, Oct.--Nov. 2006.

Digital Library

[12]

C. Cadar, D. Dunbar, and D. Engler. KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the Eighth Symposium on Operating Systems Design and Implementation (OSDI '08), pages 209--224, Dec. 2008.

Digital Library

[13]

V. Chipounov, V. Kuznetsov, and G. Candea. S2E: a platform for in-vivo multi-path analysis of software systems. In Sixteenth International Conference on Architecture Support for Programming Languages and Operating Systems (ASPLOS '11), pages 265--278, 2011.

Digital Library

[14]

A. Chou, J. Yang, B. Chelf, S. Hallem, and D. Engler. An empirical study of operating systems errors. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), pages 73--88, Nov. 2001.

Digital Library

[15]

J. Chow, B. Pfaff, T. Garfinkel, K. Christopher, and M. Rosenblum. Understanding Data Lifetime via Whole System Simulation. In Proceedings of the 13th USENIX Security Symposium, 2004.

Digital Library

[16]

J. Chow, B. Pfaff, T. Garfinkel, and M. Rosenblum. Shredding Your Garbage: Reducing Data Lifetime Through Secure Deallocation. In Proceedings of the 14th USENIX Security Symposium, 2005.

Digital Library

[17]

E. Clarke and D. Kroening. Hardware verification using ANSI-C programs as a reference. In Proceedings of ASP-DAC 2003, pages 308--311, January 2003.

Digital Library

[18]

symver:fse9A. Coen-Porisini, G. Denaro, C. Ghezzi, and M. Pezzé. Using symbolic execution for verifying safety-critical systems. In Proceedings of the Eighth European Software Engineering Conference held jointly with the Ninth ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-9), pages 142--151, 2001.

Digital Library

[19]

P. Collingbourne, C. Cadar, and P. H. Kelly. Symbolic crosschecking of floating-point and SIMD code. In Proceedings of the 6th ACM European Conference on Computer Systems (EUROSYS '11), pages 315--328, Apr. 2011.

Digital Library

[20]

Coreutils - GNU core utilities. http://www.gnu.org/software/coreutils.

[21]

M. Costa, M. Castro, L. Zhou, L. Zhang, and M. Peinado. Bouncer: securing software by blocking bad input. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP '07), pages 117--130, Oct. 2007.

Digital Library

[22]

H. Cui, J. Wu, C.-C. Tsai, and J. Yang. Stable deterministic multithreading through schedule memoization. In Proceedings of the Ninth Symposium on Operating Systems Design and Implementation (OSDI '10), Oct. 2010.

Digital Library

[23]

H. Cui, J. Wu, J. Gallagher, H. Guo, and J. Yang. Efficient deterministic multithreading through schedule relaxation. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP '11), Oct. 2011.

Digital Library

[24]

CVS. http://www.cvshome.org.

[25]

M. Das, S. Lerner, and M. Seigle. Esp: path-sensitive program verification in polynomial time. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI '02), pages 57--68, June 2002.

Digital Library

[26]

D. Engler, B. Chelf, A. Chou, and S. Hallem. Checking system rules using system-specific, programmer-written compiler extensions. In Proceedings of the Fourth Symposium on Operating Systems Design and Implementation (OSDI '00), Sept. 2000.

Digital Library

[27]

Hallem, Chou, and Chelf}sosp:deriveD. Engler, D. Yu Chen, S. Hallem, A. Chou, and B. Chelf. Bugs as deviant behavior: A general approach to inferring errors in systems code. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), 2001.

Digital Library

[28]

V. Ganesh and D. L. Dill. A decision procedure for bit-vectors and arrays. In Proceedings of the 19th International Conference On Computer Aided Verification (CAV' 07), pages 519--531, 2007.

Digital Library

[29]

X. Ge, K. Taneja, T. Xie, and N. Tillmann. Dyta: dynamic symbolic execution guided with static verification results. In Proceedings of the 33rd International Conference on Software Engineering, ICSE '11, pages 992--994, 2011.

Digital Library

[30]

Git. http://git-scm.com/.

[31]

GNU sed. http://www.gnu.org/software/sed.

[32]

P. Godefroid. Model Checking for Programming Languages using VeriSoft. In Proceedings of the 24th Annual Symposium on Principles of Programming Languages (POPL '97), pages 174--186, Jan. 1997.

Digital Library

[33]

P. Godefroid, N. Klarlund, and K. Sen. Dart: Directed automated random testing. In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI '05), pages 213--223, June 2005.

Digital Library

[34]

H. Guo, M. Wu, L. Zhou, G. Hu, J. Yang, and L. Zhang. Practical software model checking via dynamic interface reduction. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP '11), Oct. 2011.

Digital Library

[35]

B. Hackett and A. Aiken. How is aliasing used in systems software? In Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT '06/FSE-14), pages 69--80, Nov. 2006.

Digital Library

[36]

S. Hallem, B. Chelf, Y. Xie, and D. Engler. A system and language for building system-specific, static analyses. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI '02), 2002.

Digital Library

[37]

T. A. Henzinger, R. Jhala, R. Majumdar, and G. Sutre. Lazy abstraction. In Proceedings of the 29th Annual Symposium on Principles of Programming Languages, pages pp. 58--70, 2002.

Digital Library

[38]

D. Jackson and C. A. Damon. Elements of style: Analyzing a software design feature with a counterexample detector. IEEE Trans. Softw. Eng., 22 (7): 484--495, July 1996.

Digital Library

[39]

J. Jaffar, V. Murali, J. A. Navas, and A. E. Santosa. Tracer: A symbolic execution tool for verification. In Proceedings of the 24th international conference on Computer aided verification, CAV'12, July 2012.

Digital Library

[40]

R. Jhala and R. Majumdar. Path slicing. In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI '05), pages 38--47, 2005.

Digital Library

[41]

C. Killian, J. W. Anderson, R. Jhala, and A. Vahdat. Life, death, and the critical transition: Finding liveness bugs in systems code. In Proceedings of the Fourth Symposium on Networked Systems Design and Implementation (NSDI '07), pages 243--256, April 2007.

Digital Library

[42]

J. C. King. A new approach to program testing. In Proceedings of the international conference on Reliable software, pages 228--233, 1975.

Digital Library

[43]

V. Kuznetsov, J. Kinder, S. Bucur, and G. Candea. Efficient state merging in symbolic execution. In Proceedings of the ACM SIGPLAN 2012 Conference on Programming Language Design and Implementation (PLDI '12), pages 193--204, 2012.

Digital Library

[44]

K.-K. Ma, Y. P. Khoo, J. S. Foster, and M. Hicks. Directed symbolic execution. In The 18th International Static Analysis Symposium, SAS '11, Sept. 2011.

Digital Library

[45]

R. Majumdar and R.-G. Xu. Directed test generation using symbolic grammars. In Proceedings of the Seventh European Software Engineering Conference held jointly with the Seventh ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-7), pages 553--556, 2007.

Digital Library

[46]

J. S. Metos and J. V. Oldfield. Binary decision diagrams: From abstract representations to physical implementations. In DAC '83: Proceedings of the 20th conference on Design automation, pages 567--570, 1983.

Digital Library

[47]

S. Person, G. Yang, N. Rungta, and S. Khurshid. Directed incremental symbolic execution. In Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation, PLDI '11, pages 504--515, 2011.

Digital Library

[48]

C. S. Pasareanu and N. Rungta. Symbolic pathfinder: symbolic execution of java bytecode. In Proceedings of the IEEE/ACM international conference on Automated software engineering, ASE '10, pages 179--180, 2010.

Digital Library

[49]

C. S. Pasareanu, N. Rungta, and W. Visser. Symbolic execution with mixed concrete-symbolic solving. In Proceedings of the 2011 International Symposium on Software Testing and Analysis, ISSTA '11, pages 34--44, 2011.

Digital Library

[50]

QEMU. http://www.qemu.org.

[51]

D. P. Quigley, J. Sipek, C. P. Wright, and E. Zadok. UnionFS: User- and Community-oriented Development of a Unification Filesystem. In Proceedings of the 2006 Linux Symposium, volume 2, pages 349--362, Ottawa, Canada, July 2006.

[52]

D. A. Ramos and D. R. Engler. Practical, low-effort equivalence verification of real code. In Proceedings of the 23rd international conference on Computer aided verification, CAV'11, pages 669--685, 2011.

Digital Library

[53]

K. Sen, D. Marinov, and G. Agha. CUTE: A concolic unit testing engine for C. In Proceedings of the 10th European Software Engineering Conference held jointly with the 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-13), pages 263--272, Sept. 2005.

Digital Library

[54]

shadow. http://pkg-shadow.alioth.debian.org/.

[55]

S. Smith. Eat my data: How everybody gets file io wrong. In OSCON 2008, July 2008.

[56]

tar. http://www.gnu.org/software/tar/.

[57]

M. Weiser. Program slicing. In Fifth International Conference on Software Engineering, pages 439--449, 1981.

Digital Library

[58]

J. Whaley. bddbddb Project. http://bddbddb.sourceforge.net.

[59]

J. Whaley and M. S. Lam. Cloning-based context-sensitive pointer alias analysis using binary decision diagrams. In Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation (PLDI '04), pages 131--144, June 2004.

Digital Library

[60]

M. Yabandeh, N. Knezevic, D. Kostic, and V. Kuncak. CrystalBall: Predicting and preventing inconsistencies in deployed distributed systems. In Proceedings of the Sixth Symposium on Networked Systems Design and Implementation (NSDI '09), Apr. 2009.

Digital Library

[61]

J. Yang, T. Kremenek, Y. Xie, and D. Engler. MECA: an extensible, expressive system and language for statically checking security properties. In Proceedings of the 10th ACM conference on Computer and communications security (CCS '03), pages 321--334, Oct. 2003.

Digital Library

[62]

J. Yang, P. Twohey, D. Engler, and M. Musuvathi. Using model checking to find serious file system errors. In Proceedings of the Sixth Symposium on Operating Systems Design and Implementation (OSDI '04), pages 273--288, Dec. 2004.

Digital Library

[63]

J. Yang, C. Sar, and D. Engler. Explode: a lightweight, general system for finding serious storage system errors. In Proceedings of the Seventh Symposium on Operating Systems Design and Implementation (OSDI '06), pages 131--146, Nov. 2006.

Digital Library

[64]

J. Yang, C. Sar, P. Twohey, C. Cadar, and D. Engler. Automatically generating malicious disks using symbolic execution. In Proceedings of the 2006 IEEE Symposium on Security and Privacy (SP '06), pages 243--257, May 2006.

Digital Library

[65]

J. Yang, T. Chen, M. Wu, Z. Xu, X. Liu, H. Lin, M. Yang, F. Long, L. Zhang, and L. Zhou. MODIST: Transparent model checking of unmodified distributed systems. In Proceedings of the Sixth Symposium on Networked Systems Design and Implementation (NSDI '09), pages 213--228, Apr. 2009.

Digital Library

[66]

C. Zamfir and G. Candea. Execution synthesis: a technique for automated software debugging. In Proceedings of the 5th ACM European Conference on Computer Systems (EUROSYS '10), pages 321--334, Apr. 2010.

Digital Library

[67]

X. Zhang and R. Gupta. Cost effective dynamic program slicing. In Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation (PLDI '04), pages 94--106, 2004.

Digital Library

Cited By

Bose PDas DChen YFeng YKruegel CVigna G(2022)SAILFISH: Vetting Smart Contract State-Inconsistency Bugs in Seconds2022 IEEE Symposium on Security and Privacy (SP)10.1109/SP46214.2022.9833721(161-178)Online publication date: May-2022
https://doi.org/10.1109/SP46214.2022.9833721
Li HZhou LXing MTaha H(2021)Vulnerability Detection Algorithm of Lightweight Linux Internet of Things Application with Symbolic Execution Method2021 International Symposium on Computer Technology and Information Science (ISCTIS)10.1109/ISCTIS51085.2021.00013(24-27)Online publication date: Jun-2021
https://doi.org/10.1109/ISCTIS51085.2021.00013
Qian CPang LKuang XQin JZang YZhao QZhang J(2024)BSP: Branch Splitting for Unsolvable Path Hybrid FuzzingElectronics10.3390/electronics1324493513:24(4935)Online publication date: 13-Dec-2024
https://doi.org/10.3390/electronics13244935
Show More Cited By

Index Terms

Verifying systems rules using rule-directed symbolic execution

Recommendations

Verifying systems rules using rule-directed symbolic execution
ASPLOS '13

Systems code must obey many rules, such as "opened files must be closed." One approach to verifying rules is static analysis, but this technique cannot infer precise runtime effects of code, often emitting many false positives. An alternative is ...
Verifying systems rules using rule-directed symbolic execution
ASPLOS '13: Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems

Systems code must obey many rules, such as "opened files must be closed." One approach to verifying rules is static analysis, but this technique cannot infer precise runtime effects of code, often emitting many false positives. An alternative is ...
Using symbolic execution for verifying safety-critical systems

Safety critical systems require to be highly reliable and thus special care is taken when verifying them in order to increase the confidence in their behavior. This paper addresses the problem of formal verification of safety critical systems by ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGARCH Computer Architecture News

ACM SIGARCH Computer Architecture News Volume 41, Issue 1

ASPLOS '13

March 2013

540 pages

ISSN:0163-5964

DOI:10.1145/2490301

Issue’s Table of Contents

ASPLOS '13: Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems
March 2013
574 pages
ISBN:9781450318709
DOI:10.1145/2451116
General Chair:
Vivek Sarkar
Rice University, USA
,
Program Chair:
Rastislav Bodik
University of California, Berkeley, USA

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 March 2013

Published in SIGARCH Volume 41, Issue 1

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

71
Total Citations
View Citations
679
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bose PDas DChen YFeng YKruegel CVigna G(2022)SAILFISH: Vetting Smart Contract State-Inconsistency Bugs in Seconds2022 IEEE Symposium on Security and Privacy (SP)10.1109/SP46214.2022.9833721(161-178)Online publication date: May-2022
https://doi.org/10.1109/SP46214.2022.9833721
Li HZhou LXing MTaha H(2021)Vulnerability Detection Algorithm of Lightweight Linux Internet of Things Application with Symbolic Execution Method2021 International Symposium on Computer Technology and Information Science (ISCTIS)10.1109/ISCTIS51085.2021.00013(24-27)Online publication date: Jun-2021
https://doi.org/10.1109/ISCTIS51085.2021.00013
Qian CPang LKuang XQin JZang YZhao QZhang J(2024)BSP: Branch Splitting for Unsolvable Path Hybrid FuzzingElectronics10.3390/electronics1324493513:24(4935)Online publication date: 13-Dec-2024
https://doi.org/10.3390/electronics13244935
Hu QChen WWang ZLu SNie YLi XKuang X(2023)BSFuzz: Branch-State Guided Hybrid FuzzingElectronics10.3390/electronics1219403312:19(4033)Online publication date: 25-Sep-2023
https://doi.org/10.3390/electronics12194033
Han HKyea JJin YKang JPak BYun I(2023)QueryX: Symbolic Query on Decompiled Code for Finding Bugs in COTS Binaries2023 IEEE Symposium on Security and Privacy (SP)10.1109/SP46215.2023.10179314(3279-3295)Online publication date: May-2023
https://doi.org/10.1109/SP46215.2023.10179314
Zhang XGong YLiang BHuang JYou WShi WZhang JRyu SSmaragdakis Y(2022)Hunting bugs with accelerated optimal graph vertex matchingProceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3533767.3534393(64-76)Online publication date: 18-Jul-2022
https://dl.acm.org/doi/10.1145/3533767.3534393
Yi QYang G(2022)Feedback-Driven Incremental Symbolic Execution2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE55969.2022.00055(505-516)Online publication date: Oct-2022
https://doi.org/10.1109/ISSRE55969.2022.00055
Zhang GChen ZShuai ZZhang YWang J(2022)Synergizing Symbolic Execution and Fuzzing By Function-level Selective Symbolization2022 29th Asia-Pacific Software Engineering Conference (APSEC)10.1109/APSEC57359.2022.00045(328-337)Online publication date: Dec-2022
https://doi.org/10.1109/APSEC57359.2022.00045
Pan WChen ZZhang GLuo YZhang YWang JCadar CZhang X(2021)Grammar-agnostic symbolic execution by token symbolizationProceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3460319.3464845(374-387)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3460319.3464845
Liu ZZhu SQin BChen HSong LSherwood TBerger EKozyrakis C(2021)Automatically detecting and fixing concurrency bugs in go software systemsProceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3445814.3446756(616-629)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3445814.3446756
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents