research-article

Open access

Synthesizing contracts correct modulo a test generator

Authors:

Angello Astorga,

Shambwaditya Saha,

Tao XieAuthors Info & Claims

Proceedings of the ACM on Programming Languages, Volume 5, Issue OOPSLA

Article No.: 104, Pages 1 - 27

https://doi.org/10.1145/3485481

Published: 15 October 2021 Publication History

Abstract

We present an approach to learn contracts for object-oriented programs where guarantees of correctness of the contracts are made with respect to a test generator. Our contract synthesis approach is based on a novel notion of tight contracts and an online learning algorithm that works in tandem with a test generator to synthesize tight contracts. We implement our approach in a tool called Precis and evaluate it on a suite of programs written in C#, studying the safety and strength of the synthesized contracts, and compare them to those synthesized by Daikon.

References

[1]

Rajeev Alur, Rastislav Bodík, Eric Dallal, Dana Fisman, Pranav Garg, Garvit Juniwal, Hadas Kress-Gazit, P. Madhusudan, Milo M. K. Martin, Mukund Raghothaman, Shambwaditya Saha, Sanjit A. Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. 2015. Syntax-guided synthesis. In Dependable Software Systems Engineering 2015.

[2]

Rajeev Alur, Arjun Radhakrishna, and Abhishek Udupa. 2017. Scaling enumerative program synthesis via divide and conquer. In Tools and Algorithms for the Construction and Analysis of Systems.

[3]

Rajeev Alur, Pavol Černý, P. Madhusudan, and Wonhong Nam. 2005. Synthesis of interface specifications for Java classes. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. https://doi.org/10.1145/1040305.1040314

Digital Library

[4]

Glenn Ammons, Rastislav Bodík, and James R. Larus. 2002. Mining Specifications. In Proceedings of the 29th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’02). Association for Computing Machinery, New York, NY, USA. isbn:1581134509 https://doi.org/10.1145/503272.503275

Digital Library

[5]

Angello Astorga, P. Madhusudan, Shambwaditya Saha, Shiyu Wang, and Tao Xie. 2019. Learning Stateful Preconditions modulo a Test Generator. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2019). Association for Computing Machinery, New York, NY, USA. isbn:9781450367127 https://doi.org/10.1145/3314221.3314641

Digital Library

[6]

A. Astorga, S. Srisakaokul, X. Xiao, and T. Xie. 2018. PreInfer: Automatic Inference of Preconditions via Symbolic Analysis. In 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 678–689. https://doi.org/10.1109/DSN.2018.00074

[7]

Mike Barnett, K. Rustan M. Leino, and Wolfram Schulte. 2005. The Spec# Programming System: An Overview. In Construction and Analysis of Safe, Secure, and Interoperable Smart Devices.

[8]

Adam Betts, Nathan Chong, Alastair F. Donaldson, Shaz Qadeer, and Paul Thomson. 2012. GPUVerify: a verifier for GPU kernels. In Proceedings of the 27th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2012, part of SPLASH 2012, Tucson, AZ, USA, October 21-25, 2012, Gary T. Leavens and Matthew B. Dwyer (Eds.). ACM, 113–132. https://doi.org/10.1145/2384616.2384625

Digital Library

[9]

Feng Chen and Grigore Roşu. 2007. Mop: An Efficient and Generic Runtime Verification Framework. SIGPLAN Not., 42, 10 (2007), Oct., issn:0362-1340 https://doi.org/10.1145/1297105.1297069

Digital Library

[10]

Berkeley Churchill, Oded Padon, Rahul Sharma, and Alex Aiken. 2019. Semantic Program Alignment for Equivalence Checking. In PLDI 2019.

[11]

Patrick Cousot, Radhia Cousot, Manuel Fähndrich, and Francesco Logozzo. 2013. Automatic Inference of Necessary Preconditions. In Verification, Model Checking, and Abstract Interpretation, Roberto Giacobazzi, Josh Berdine, and Isabella Mastroeni (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. isbn:978-3-642-35873-9

[12]

Christoph Csallner, Nikolai Tillmann, and Yannis Smaragdakis. 2008. DySy: Dynamic symbolic execution for invariant inference. In ICSE 2008.

Digital Library

[13]

Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In TACAS 2008.

Digital Library

[14]

Daniel DeFreez, Haaken Martinson Baldwin, Cindy Rubio-González, and Aditya V. Thakur. 2019. Effective error-specification inference via domain-knowledge expansion. In FSE 2019.

[15]

Isil Dillig, Thomas Dillig, Boyang Li, and Ken McMillan. 2013. Inductive invariant generation via abductive inference. In OOPSLA 2013.

Digital Library

[16]

Nii Dodoo, Lin Li, and Michael Ernst. 2003. Selecting, Refining, and Evaluating Predicates for Program Analysis.

[17]

Michael D. Ernst. 2000. Dynamically Discovering Likely Program Invariants. University of Washington Department of Computer Science and Engineering. Seattle, Washington.

[18]

Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin. 1999. Dynamically Discovering Likely Program Invariants to Support Program Evolution. In ICSE 1999.

Digital Library

[19]

P. Ezudheen, Daniel Neider, Deepak D’Souza, Pranav Garg, and P. Madhusudan. 2018. Horn-ICE learning for synthesizing invariants and contracts. In OOPSLA 2018.

[20]

Manuel Fähndrich. 2010. Static Verification for Code Contracts. In SAS 2010.

[21]

Robert W. Floyd. 1960. An Algorithm Defining ALGOL Assignment Statements. Commun. ACM, 3, 3 (1960), 170–171. https://doi.org/10.1145/367149.367170

Digital Library

[22]

Gordon Fraser and Andreas Zeller. 2011. Generating Parameterized Unit Tests. In Proceedings of the 2011 International Symposium on Software Testing and Analysis (ISSTA ’11). Association for Computing Machinery, New York, NY, USA. isbn:9781450305624 https://doi.org/10.1145/2001420.2001464

Digital Library

[23]

Pranav Garg, Daniel Neider, P. Madhusudan, and Dan Roth. 2016. Learning Invariants Using Decision Trees and Implication Counterexamples. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’16). Association for Computing Machinery, New York, NY, USA. isbn:9781450335492 https://doi.org/10.1145/2837614.2837664

Digital Library

[24]

Milos Gligoric, Tihomir Gvero, Vilas Jagannath, Sarfraz Khurshid, Viktor Kuncak, and Darko Marinov. 2010. Test Generation through Programming in UDITA. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1. https://doi.org/10.1145/1806799.1806835

Digital Library

[25]

Thomas A. Henzinger, Ranjit Jhala, and Rupak Majumdar. 2005. Permissive Interfaces. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-13). https://doi.org/10.1145/1081706.1081713

Digital Library

[26]

Charles Antony Richard Hoare. 1969. An axiomatic basis for computer programming. Commun. ACM, 12, 10 (1969), 576–580.

Digital Library

[27]

Gunel Jahangirova, David Clark, Mark Harman, and Paolo Tonella. 2016. Test Oracle Assessment and Improvement. In ISSTA 2016.

[28]

Ton Chanh Le, Guolong Zheng, and ThanhVu Nguyen. 2019. SLING: Using Dynamic Analysis to Infer Program Invariants in Separation Logic. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2019). https://doi.org/10.1145/3314221.3314634

Digital Library

[29]

Gary T. Leavens, Albert L. Baker, and Clyde Ruby. 2006. Preliminary Design of JML: A Behavioral Interface Specification Language for Java. SIGSOFT Softw. Eng. Notes.

[30]

Bertrand Meyer. 1988. Object-Oriented Software Construction (1st ed.). Prentice-Hall, Inc., USA. isbn:0136290493

Digital Library

[31]

Thomas M. Mitchell. 1997. Machine Learning (1 ed.).

[32]

Facundo Molina, Pablo Ponzio, Nazareno Aguirre, and Marcelo F. Frias. 2021. EvoSpex: An Evolutionary Algorithm for Learning Postconditions. arxiv:2102.13569.

[33]

Mary M. Moya and Don R. Hush. 1996. Network constraints and multi-objective optimization for one-class classification. Neural Networks, 9, 3 (1996), 463–474. http://dblp.uni-trier.de/db/journals/nn/nn9.html##MoyaH96

Digital Library

[34]

Daniel Neider, Shambwaditya Saha, and P. Madhusudan. 2016. Synthesizing Piece-Wise Functions by Learning Classifiers. In Proceedings of the 22nd International Conference on Tools and Algorithms for the Construction and Analysis of Systems - Volume 9636. https://doi.org/10.1007/978-3-662-49674-9_11

Digital Library

[35]

Julie L. Newcomb, Andrew Adams, Steven Johnson, Rastislav Bodik, and Shoaib Kamil. 2020. Verifying and Improving Halide’s Term Rewriting System with Program Synthesis. Proc. ACM Program. Lang., 4, OOPSLA (2020), Article 166, Nov., https://doi.org/10.1145/3428234

Digital Library

[36]

Pengyu Nie, Marinela Parovic, Zhiqiang Zang, Sarfraz Khurshid, Aleksandar Milicevic, and Milos Gligoric. 2020. Unifying Execution of Imperative Generators and Declarative Specifications. Proc. ACM Program. Lang., 4, OOPSLA (2020), Article 217, Nov., https://doi.org/10.1145/3428285

Digital Library

[37]

Carlos Pacheco and Michael D. Ernst. 2007. Randoop: Feedback-Directed Random Testing for Java. In Companion to the 22nd ACM SIGPLAN Conference on Object-Oriented Programming Systems and Applications Companion (OOPSLA ’07). https://doi.org/10.1145/1297846.1297902

Digital Library

[38]

Saswat Padhi, Rahul Sharma, and Todd Millstein. 2016. Data-Driven Precondition Inference with Learned Features. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’16). Association for Computing Machinery, New York, NY, USA. isbn:9781450342612 https://doi.org/10.1145/2908080.2908099

Digital Library

[39]

Rahul Pandita, Xusheng Xiao, Hao Zhong, Tao Xie, Stephen Oney, and Amit Paradkar. 2012. Inferring Method Specifications from Natural Language API Descriptions. In 2012 34th International Conference on Software Engineering (ICSE).

[40]

Nadia Polikarpova, Ilinca Ciupa, and Bertrand Meyer. 2009. A Comparative Study of Programmer-Written and Automatically Inferred Contracts. In Proceedings of the Eighteenth International Symposium on Software Testing and Analysis (ISSTA ’09). Association for Computing Machinery, New York, NY, USA. isbn:9781605583389 https://doi.org/10.1145/1572272.1572284

Digital Library

[41]

Andrew Reynolds, Haniel Barbosa, Andres Nötzli, Clark Barrett, and Cesare Tinelli. 2019. cvc4sy: Smart and Fast Term Enumeration for Syntax-Guided Synthesis. In Computer Aided Verification.

[42]

Todd W. Schiller, Kellen Donohue, Forrest Coward, and Michael D. Ernst. 2014. Case Studies and Tools for Contract Specifications. In Proceedings of the 36th International Conference on Software Engineering (ICSE 2014). https://doi.org/10.1145/2568225.2568285

Digital Library

[43]

J. M. Spivey. 1988. Understanding Z: A Specification Language and Its Formal Semantics. Cambridge University Press, USA. isbn:0521334292

Digital Library

[44]

Valerio Terragni, Gunel Jahangirova, Paolo Tonella, and Mauro Pezzè. 2020. Evolutionary Improvement of Assertion Oracles. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020). https://doi.org/10.1145/3368089.3409758

Digital Library

[45]

Nikolai Tillmann and Jonathan De Halleux. 2008. Pex: White Box Test Generation for .NET. In Tests and Proofs.

[46]

John Whaley, Michael C. Martin, and Monica S. Lam. 2002. Automatic Extraction of Object-Oriented Component Interfaces. In Proceedings of the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’02). https://doi.org/10.1145/566172.566212

Digital Library

[47]

Tao Xie, Evan Martin, and Hai Yuan. 2006. Automatic Extraction of Abstract-Object-State Machines from Unit-Test Executions. In Proceedings of the 28th International Conference on Software Engineering (ICSE ’06). Association for Computing Machinery, New York, NY, USA. isbn:1595933751 https://doi.org/10.1145/1134285.1134427

Digital Library

[48]

Juan Zhai, Yu Shi, Minxue Pan, Guian Zhou, Yongxiang Liu, Chunrong Fang, Shiqing Ma, Lin Tan, and Xiangyu Zhang. 2020. C2S: Translating Natural Language Comments to Formal Program Specifications. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2020). https://doi.org/10.1145/3368089.3409716

Digital Library

[49]

He Zhu, Stephen Magill, and Suresh Jagannathan. 2018. A Data-Driven CHC Solver. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2018). https://doi.org/10.1145/3192366.3192416

Digital Library

Cited By

Astorga AHsieh CMadhusudan PMitra S(2023)Perception Contracts for Safety of ML-Enabled SystemsProceedings of the ACM on Programming Languages10.1145/36228757:OOPSLA2(2196-2223)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3622875
Park KD'Antoni LReps T(2023)Synthesizing SpecificationsProceedings of the ACM on Programming Languages10.1145/36228617:OOPSLA2(1787-1816)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3622861
Krogmeier PMadhusudan P(2023)Languages with Decidable Learning: A Meta-theoremProceedings of the ACM on Programming Languages10.1145/35860327:OOPSLA1(143-171)Online publication date: 6-Apr-2023
https://dl.acm.org/doi/10.1145/3586032
Show More Cited By

Index Terms

Synthesizing contracts correct modulo a test generator

Recommendations

Learning stateful preconditions modulo a test generator
PLDI 2019: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation

In this paper, we present a novel learning framework for inferring stateful preconditions (i.e., preconditions constraining not only primitive-type inputs but also non-primitive-type object states) modulo a test generator, where the quality of the ...
Synthesising implicit contracts
ICAIL '13: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Law

In regulated interactive systems, one party's behaviour may impose restrictions on how others may behave when interacting with it. These restrictions may be seen as implicit contracts which the affected party has to conform to and may thus be considered ...
Correct blame for contracts: no more scapegoating
POPL '11: Proceedings of the 38th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages

Behavioral software contracts supplement interface information with logical assertions. A rigorous enforcement of contracts provides useful feedback to developers if it signals contract violations as soon as they occur and if it assigns blame to ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages

Proceedings of the ACM on Programming Languages Volume 5, Issue OOPSLA

October 2021

2001 pages

EISSN:2475-1421

DOI:10.1145/3492349

Issue’s Table of Contents

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2021

Published in PACMPL Volume 5, Issue OOPSLA

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF (National Science Foundation)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
367
Total Downloads

Downloads (Last 12 months)129
Downloads (Last 6 weeks)8

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Astorga AHsieh CMadhusudan PMitra S(2023)Perception Contracts for Safety of ML-Enabled SystemsProceedings of the ACM on Programming Languages10.1145/36228757:OOPSLA2(2196-2223)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3622875
Park KD'Antoni LReps T(2023)Synthesizing SpecificationsProceedings of the ACM on Programming Languages10.1145/36228617:OOPSLA2(1787-1816)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3622861
Krogmeier PMadhusudan P(2023)Languages with Decidable Learning: A Meta-theoremProceedings of the ACM on Programming Languages10.1145/35860327:OOPSLA1(143-171)Online publication date: 6-Apr-2023
https://dl.acm.org/doi/10.1145/3586032
Kalita PMuduli SD’Antoni LReps TRoy S(2022)Synthesizing abstract transformersProceedings of the ACM on Programming Languages10.1145/35633346:OOPSLA2(1291-1319)Online publication date: 31-Oct-2022
https://dl.acm.org/doi/10.1145/3563334

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents