Mostly Automated Proof Repair for Verified Libraries
Article No.: 107, Pages 25 - 49
Abstract
The cost of maintaining formally specified and verified software is widely considered prohibitively high due to the need to constantly keep code and the proofs of its correctness in sync—the problem known as proof repair. One of the main challenges in automated proof repair for evolving code is to infer invariants for a new version of a once verified program that are strong enough to establish its full functional correctness.
In this work, we present the first proof repair methodology for higher-order imperative functions, whose initial versions were verified in the Coq proof assistant and whose specifications remained unchanged. Our proof repair procedure is based on the combination of dynamic program alignment, enumerative invariant synthesis, and a novel technique for efficiently pruning the space of invariant candidates, dubbed proof-driven testing, enabled by the constructive nature of Coq’s proof certificates.
We have implemented our approach in a mostly-automated proof repair tool called Sisyphus. Given an OCaml function verified in Coq and its unverified new version, Sisyphus produces a Coq proof for the new version, discharging most of the new proof goals automatically and suggesting high-confidence obligations for the programmer to prove for the cases when automation fails. We have evaluated Sisyphus on 10 OCaml programs taken from popular libraries, that manipulate arrays and mutable data structures, considering their verified original and unverified evolved versions. Sisyphus has managed to repair proofs for all those functions, suggesting correct invariants and generating a small number of easy-to-prove residual obligations.
References
[1]
Mark Adams. 2015. Refactoring Proofs with Tactician. In SEFM (LNCS, Vol. 9509). Springer, 53–67. https://doi.org/10.1007/978-3-662-49224-6_6
[2]
Wolfgang Ahrendt, Christoph Gladisch, and Mihai Herda. 2016. Proof-based Test Case Generation. In Deductive Software Verification - The KeY Book - From Theory to Practice (LNCS, Vol. 10001). Springer, 415–451. https://doi.org/10.1007/978-3-319-49812-6_12
[3]
Andrew W. Appel. 2011. Verified Software Toolchain - (Invited Talk). In ESOP (LNCS, Vol. 6602). Springer, 1–17. https://doi.org/10.1007/978-3-642-19718-5_1
[4]
Andrew W. Appel and David A. Naumann. 2020. Verified sequential Malloc/Free. In ISMM. ACM, 48–59. https://doi.org/10.1145/3381898.3397211
[5]
Timothy Bourke, Matthias Daum, Gerwin Klein, and Rafal Kolanski. 2012. Challenges and Experiences in Managing Large-Scale Proofs. In 11th International Conference Intelligent Computer Mathematics (CICM) (LNCS, Vol. 7362). Springer, 32–48. https://doi.org/10.1007/978-3-642-31374-5_3
[6]
Marc Brockschmidt, Yuxin Chen, Pushmeet Kohli, Siddharth Krishna, and Daniel Tarlow. 2017. Learning Shape Analysis. In SAS (LNCS, Vol. 10422). Springer, 66–87. https://doi.org/10.1007/978-3-319-66706-5_4
[7]
Arthur Charguéraud. 2011. Characteristic Formulae for the Verification of Imperative Programs. In ICFP. ACM, 418–430. https://doi.org/10.1145/2034773.2034828
[8]
Arthur Charguéraud. 2020. Separation Logic for Sequential Programs (Functional Pearl). Proc. ACM Program. Lang., 4, ICFP (2020), 116:1–116:34. https://doi.org/10.1145/3408998
[9]
Arthur Charguéraud, Jean-Christophe Filliâtre, François Pottier, and Mário Pereira. 2017. VOCAL – A Verified OCaml Library. In ML Family Workshop.
[10]
Haogang Chen, Daniel Ziegler, Tej Chajed, Adam Chlipala, M. Frans Kaashoek, and Nickolai Zeldovich. 2015. Using Crash Hoare logic for certifying the FSCQ file system. In SOSP. ACM, 18–37. https://doi.org/10.1145/2815400.2815402
[11]
Adam Chlipala. 2011. Mostly-automated verification of low-level programs in computational separation logic. In PLDI. ACM, 234–245. https://doi.org/10.1145/1993498.1993526
[12]
Satyaki Das, David L. Dill, and Seungjoon Park. 1999. Experience with Predicate Abstraction. In CAV, Nicolas Halbwachs and Doron A. Peled (Eds.) (LNCS, Vol. 1633). Springer, 160–171. https://doi.org/10.1007/3-540-48683-6_16
[13]
Leonardo Mendonça de Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In TACAS (LNCS, Vol. 4963). Springer, 337–340. https://doi.org/10.1007/978-3-540-78800-3_24
[14]
Richard A. DeMillo, Richard J. Lipton, and Alan J. Perlis. 1977. Social Processes and Proofs of Theorems and Programs. In POPL. ACM, 206–214. https://doi.org/10.1145/512950.512970
[15]
Andres Erbsen, Jade Philipoom, Jason Gross, Robert Sloan, and Adam Chlipala. 2019. Simple High-Level Code for Cryptographic Arithmetic - With Proofs, Without Compromises. In IEEE Symposium on Security and Privacy. IEEE. https://doi.org/10.1109/SP.2019.00005
[16]
Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin. 1999. Dynamically Discovering Likely Program Invariants to Support Program Evolution. In ICSE. ACM, 213–224. https://doi.org/10.1145/302405.302467
[17]
Michael D. Ernst, Adam Czeisler, William G. Griswold, and David Notkin. 2000. Quickly detecting relevant program invariants. In ICSE. ACM, 449–458. https://doi.org/10.1145/337180.337240
[18]
Jean-Christophe Filliâtre and Andrei Paskevich. 2013. Why3 - Where Programs Meet Provers. In ESOP (LNCS, Vol. 7792). Springer, 125–128. https://doi.org/10.1007/978-3-642-37036-6_8
[19]
Cormac Flanagan and K. Rustan M. Leino. 2001. Houdini, an Annotation Assistant for ESC/Java. In FME (LNCS, Vol. 2021). Springer, 500–517. https://doi.org/10.1007/3-540-45251-6_29
[20]
Robert W. Floyd. 1967. Assigning Meanings to Programs. Proceedings of Symposium on Applied Mathematics, 19 (1967), 19–32.
[21]
Georges Gonthier, Beta Ziliani, Aleksandar Nanevski, and Derek Dreyer. 2011. How to make ad hoc proof automation less ad hoc. In ICFP. ACM, 163–175. https://doi.org/10.1145/2034773.2034798
[22]
Kiran Gopinathan, Mayank Keoliya, and Ilya Sergey. 2023. Reproduction Artefact for Article “Mostly Automated Proof Repair for Verified Libraries". https://doi.org/10.5281/zenodo.7703886
[23]
Susanne Graf and Hassen Saïdi. 1997. Construction of Abstract State Graphs with PVS. In CAV (LNCS, Vol. 1254). Springer, 72–83. https://doi.org/10.1007/3-540-63166-6_10
[24]
Ronghui Gu, Zhong Shao, Hao Chen, Xiongnan (Newman) Wu, Jieung Kim, Vilhelm Sjöberg, and David Costanzo. 2016. CertiKOS: An Extensible Architecture for Building Certified Concurrent OS Kernels. In OSDI. USENIX Association, 653–669.
[25]
Chris Hawblitzel, Jon Howell, Manos Kapritsos, Jacob R. Lorch, Bryan Parno, Michael L. Roberts, Srinath T. V. Setty, and Brian Zill. 2015. IronFleet: proving practical distributed systems correct. In SOSP. ACM, 1–17. https://doi.org/10.1145/2815400.2815428
[26]
Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Kenneth L. McMillan. 2004. Abstractions from proofs. In POPL. ACM, 232–244. https://doi.org/10.1145/964001.964021
[27]
C. A. R. Hoare. 1969. An Axiomatic Basis for Computer Programming. Commun. ACM, 12, 10 (1969), 576–580. https://doi.org/10.1145/363235.363259
[28]
The Iris Project. 2022. Iris: a Higher-Order Concurrent Separation Logic Framework, implemented and verified in the Coq proof assistant. https://iris-project.org/ Online; last accessed 6 November 2022
[29]
Bart Jacobs, Jan Smans, Pieter Philippaerts, Frédéric Vogels, Willem Penninckx, and Frank Piessens. 2011. VeriFast: A Powerful, Sound, Predictable, Fast Verifier for C and Java. In NASA Formal Methods (LNCS, Vol. 6617). Springer, 41–55.
[30]
Gerwin Klein, Kevin Elphinstone, Gernot Heiser, June Andronick, David Cock, Philip Derrin, Dhammika Elkaduwe, Kai Engelhardt, Rafal Kolanski, Michael Norrish, Thomas Sewell, Harvey Tuch, and Simon Winwood. 2009. seL4: formal verification of an OS kernel. In SOSP. ACM, 207–220. https://doi.org/10.1145/1629575.1629596
[31]
Ramana Kumar, Magnus O. Myreen, Michael Norrish, and Scott Owens. 2014. CakeML: a verified implementation of ML. In POPL. ACM, 179–192. https://doi.org/10.1145/2535838.2535841
[32]
Ton Chanh Le, Guolong Zheng, and ThanhVu Nguyen. 2019. SLING: Using Dynamic Analysis to Infer Program Invariants in Separation Logic. In PLDI. ACM, 788–801. https://doi.org/10.1145/3314221.3314634
[33]
K. Rustan M. Leino. 2010. Dafny: An Automatic Program Verifier for Functional Correctness. In LPAR (LNCS, Vol. 6355). Springer, 348–370. https://doi.org/10.1007/978-3-642-17511-4_20
[34]
Xavier Leroy. 2006. Formal certification of a compiler back-end or: programming a compiler with a proof assistant. In POPL. ACM, 42–54. https://doi.org/10.1145/1111037.1111042
[35]
Pierre Letouzey. 2008. Extraction in Coq: An Overview. In 4th Conference on Computability in Europe (CiE) (LNCS, Vol. 5028). Springer, 359–369. https://doi.org/10.1007/978-3-540-69407-6_39
[36]
Stephen Magill, Aleksandar Nanevski, Edmund Clarke, and Peter Lee. 2006. Inferring Invariants in Separation Logic for Imperative List-Processing Programs. The third workshop on Semantics, Program Analysis and Computing Environments for Memory Management (SPACE), 1, 1, 5–7.
[37]
Daniel Matichuk. 2012. Automatic Function Annotations for Hoare Logic. In Proceedings Seventh Conference on Systems Software Verification (SSV) (EPTCS, Vol. 102). 46–56. https://doi.org/10.4204/EPTCS.102.6
[38]
Ike Mulder, Robbert Krebbers, and Herman Geuvers. 2022. Diaframe: automated verification of fine-grained concurrent programs in Iris. In PLDI. ACM, 809–824. https://doi.org/10.1145/3519939.3523432
[39]
Peter Müller, Malte Schwerhoff, and Alexander J. Summers. 2016. Viper: A Verification Infrastructure for Permission-Based Reasoning. In VMCAI (LNCS, Vol. 9583). Springer, 41–62. https://doi.org/10.1007/978-3-662-49122-5_2
[40]
Aleksandar Nanevski, Viktor Vafeiadis, and Josh Berdine. 2010. Structuring the verification of heap-manipulating programs. In POPL. 261–274. https://doi.org/10.1145/1706299.1706331
[41]
Peter W. O’Hearn, John C. Reynolds, and Hongseok Yang. 2001. Local Reasoning about Programs that Alter Data Structures. In CSL (LNCS, Vol. 2142). Springer, 1–19. https://doi.org/10.1007/3-540-44802-0_1
[42]
Guillaume Petiot, Bernard Botella, Jacques Julliand, Nikolai Kosmatov, and Julien Signoles. 2014. Instrumentation of Annotated C Programs for Test Generation. In 14th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM). IEEE Computer Society, 105–114. https://doi.org/10.1109/SCAM.2014.19
[43]
Ruzica Piskac, Thomas Wies, and Damien Zufferey. 2014. GRASShopper - Complete Heap Verification with Mixed Specifications. In TACAS (LNCS, Vol. 8413). Springer, 124–139. https://doi.org/10.1007/978-3-642-54862-8_9
[44]
Nadia Polikarpova, Julian Tschannen, and Carlo A. Furia. 2018. A fully verified container library. Formal Aspects Comput., 30, 5 (2018), 495–523. https://doi.org/10.1007/s00165-017-0435-1
[45]
Shengchao Qin, Guanhua He, Chenguang Luo, Wei-Ngan Chin, and Xin Chen. 2013. Loop invariant synthesis in a combined abstract domain. J. Symb. Comput., 50 (2013), 386–408. https://doi.org/10.1016/j.jsc.2012.08.007
[46]
Vincent Rahli, Ivana Vukotic, Marcus Völp, and Paulo Jorge Esteves Veríssimo. 2018. Velisarios: Byzantine Fault-Tolerant Protocols Powered by Coq. In ESOP, Amal Ahmed (Ed.) (LNCS, Vol. 10801). Springer, 619–650. https://doi.org/10.1007/978-3-319-89884-1_22
[47]
John C. Reynolds. 2002. Separation Logic: A Logic for Shared Mutable Data Structures. In LICS. IEEE Computer Society, 55–74. https://doi.org/10.1109/LICS.2002.1029817
[48]
Talia Ringer. 2021. Proof Repair. Ph. D. Dissertation. University of Washington, USA.
[49]
Talia Ringer, Karl Palmskog, Ilya Sergey, Milos Gligoric, and Zachary Tatlock. 2019. QED at Large: A Survey of Engineering of Formally Verified Software. Found. Trends Program. Lang., 5, 2-3 (2019), 102–281. https://doi.org/10.1561/2500000045
[50]
Talia Ringer, RanDair Porter, Nathaniel Yazdani, John Leo, and Dan Grossman. 2021. Proof repair across type equivalences. In PLDI. ACM, 112–127. https://doi.org/10.1145/3453483.3454033
[51]
Talia Ringer, Nathaniel Yazdani, John Leo, and Dan Grossman. 2018. Adapting proof automation to adapt proofs. In CPP. ACM, 115–129. https://doi.org/10.1145/3167094
[52]
Talia Ringer, Nathaniel Yazdani, John Leo, and Dan Grossman. 2019. Ornaments for Proof Reuse in Coq. In ITP (LIPIcs, Vol. 141). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 26:1–26:19. https://doi.org/10.4230/LIPIcs.ITP.2019.26
[53]
Valentin Robert. 2018. Front-end tooling for building and maintaining dependently-typed functional programs. Ph. D. Dissertation. University of California, San Diego, USA.
[54]
Michael Sammler, Rodolphe Lepigre, Robbert Krebbers, Kayvan Memarian, Derek Dreyer, and Deepak Garg. 2021. RefinedC: automating the foundational verification of C code with refined ownership types. In PLDI. ACM, 158–174. https://doi.org/10.1145/3453483.3454036
[55]
Ilya Sergey, Aleksandar Nanevski, and Anindya Banerjee. 2015. Mechanized verification of fine-grained concurrent programs. In PLDI. ACM, 77–87. https://doi.org/10.1145/2737924.2737964
[56]
Xujie Si, Hanjun Dai, Mukund Raghothaman, Mayur Naik, and Le Song. 2018. Learning Loop Invariants for Program Verification. In NeurIPS. 7762–7773.
[57]
Amin Timany and Bart Jacobs. 2015. First Steps Towards Cumulative Inductive Types in CIC. In ICTAC (LNCS). Springer, 608–617. https://doi.org/10.1007/978-3-319-25150-9_36
[58]
Karin Wibergh. 2019. Automatic refactoring for Agda. Master’s thesis. Chalmers University of Technology and University of Gothenburg.
[59]
James R. Wilcox, Doug Woos, Pavel Panchekha, Zachary Tatlock, Xi Wang, Michael D. Ernst, and Thomas E. Anderson. 2015. Verdi: a framework for implementing and formally verifying distributed systems. In PLDI. ACM, 357–368. https://doi.org/10.1145/2737924.2737958
[60]
Doug Woos, James R. Wilcox, Steve Anton, Zachary Tatlock, Michael D. Ernst, and Thomas E. Anderson. 2016. Planning for change in a formal verification of the raft consensus protocol. ACM, 154–165. https://doi.org/10.1145/2854065.2854081
Index Terms
- Mostly Automated Proof Repair for Verified Libraries
Recommendations
Verified heap theorem prover by paramodulation
ICFP '12We present VeriStar, a verified theorem prover for a decidable subset of separation logic. Together with VeriSmall [3], a proved-sound Smallfoot-style program analysis for C minor, VeriStar demonstrates that fully machine-checked static analyses ...
Certifying the synthesis of heap-manipulating programs
Automated deductive program synthesis promises to generate executable programs from concise specifications, along with proofs of correctness that can be independently verified using third-party tools. However, an attempt to exercise this promise using ...
Verified heap theorem prover by paramodulation
ICFP '12: Proceedings of the 17th ACM SIGPLAN international conference on Functional programmingWe present VeriStar, a verified theorem prover for a decidable subset of separation logic. Together with VeriSmall [3], a proved-sound Smallfoot-style program analysis for C minor, VeriStar demonstrates that fully machine-checked static analyses ...
Comments
Information & Contributors
Information
Published In
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 June 2023
Published in PACMPL Volume 7, Issue PLDI
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Funding Sources
- Singapore Ministry of Education (MoE)
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 377Total Downloads
- Downloads (Last 12 months)274
- Downloads (Last 6 weeks)15
Reflects downloads up to 18 Aug 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in