Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article
Public Access

Program synthesis using conflict-driven learning

Published: 11 June 2018 Publication History

Abstract

We propose a new conflict-driven program synthesis technique that is capable of learning from past mistakes. Given a spurious program that violates the desired specification, our synthesis algorithm identifies the root cause of the conflict and learns new lemmas that can prevent similar mistakes in the future. Specifically, we introduce the notion of equivalence modulo conflict and show how this idea can be used to learn useful lemmas that allow the synthesizer to prune large parts of the search space. We have implemented a general-purpose CDCL-style program synthesizer called Neo and evaluate it in two different application domains, namely data wrangling in R and functional programming over lists. Our experiments demonstrate the substantial benefits of conflict-driven learning and show that Neo outperforms two state-of-the-art synthesis tools, Morpheus and Deepcoder, that target these respective domains.

Supplementary Material

WEBM File (p420-feng.webm)

References

[1]
Aws Albarghouthi, Sumit Gulwani, and Zachary Kincaid. 2013. Recursive Program Synthesis. In Proc. International Conference on Computer Aided Verification . Springer, 934–950.
[2]
Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo MK Martin, Mukund Raghothaman, Sanjit A Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. 2013. Syntax-guided synthesis. In Proc. Formal Methods in Computer-Aided Design. IEEE, 1–8.
[3]
Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2017. Deepcoder: Learning to write programs. In Proc. International Conference on Learning Representations . OpenReview.
[4]
Daniel Le Berre and Anne Parrain. 2010. The Sat4j library, release 2.2. Journal on Satisfiability, Boolean Modeling and Computation (2010), 59–6.
[5]
Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh. 2009. Conflict-driven clause learning SAT solvers. Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications (2009), 131–153.
[6]
Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In Proc. Tools and Algorithms for Construction and Analysis of Systems . Springer, 337–340.
[7]
Yu Feng, Ruben Martins, Osbert Bastani, and Isil Dillig. 2018. Neo. http://utopia-group.github.io/neo/ .
[8]
Yu Feng, Ruben Martins, Jacob Van Geffen, Isil Dillig, and Swarat Chaudhuri. 2017. Component-based synthesis of table consolidation and transformation tasks from examples. In Proc. Conference on Programming Language Design and Implementation . ACM, 422–436.
[9]
Yu Feng, Ruben Martins, Yuepeng Wang, Isil Dillig, and Thomas Reps. 2017. Component-Based Synthesis for Complex APIs. In Proc. Symposium on Principles of Programming Languages . ACM, 599–612.
[10]
John K. Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing Data Structure Transformations from Input-output Examples. In Proc. Conference on Programming Language Design and Implementation . ACM, 229–239.
[11]
Harald Ganzinger, George Hagen, Robert Nieuwenhuis, Albert Oliveras, and Cesare Tinelli. 2004. DPLL( T): Fast Decision Procedures. In Proc. International Conference on Computer Aided Verification . Springer, 175–188.
[12]
Adrià Gascón, Ashish Tiwari, Brent Carmer, and Umang Mathur. 2017. Look for the Proof to Find the Program: Decorated-Component-Based Program Synthesis. In Proc. International Conference on Computer Aided Verification . Springer, 86–103.
[13]
Sumit Gulwani. 2011. Automating string processing in spreadsheets using input-output examples. In Proc. Symposium on Principles of Programming Languages . ACM, 317–330.
[14]
Sumit Gulwani, Susmit Jha, Ashish Tiwari, and Ramarathnam Venkatesan. 2011. Synthesis of loop-free programs. In Proc. Conference on Programming Language Design and Implementation . ACM, 62–73.
[15]
Tihomir Gvero, Viktor Kuncak, Ivan Kuraj, and Ruzica Piskac. 2013. Complete completion using types and weights. In Proc. Conference on Programming Language Design and Implementation . ACM, 27–38.
[16]
Susmit Jha, Sumit Gulwani, Sanjit A. Seshia, and Ashish Tiwari. 2010. Oracle-guided component-based program synthesis. In Proc. International Conference on Software Engineering . ACM/IEEE, 215–224.
[17]
David Mandelin, Lin Xu, Rastislav Bodík, and Doug Kimelman. 2005. Jungloid mining: helping to navigate the API jungle. In Proc. Conference on Programming Language Design and Implementation . ACM, 48–61.
[18]
Joao Marques-Silva and Karem A. Sakallah. 1999. GRASP: A Search Algorithm for Propositional Satisfiability. IEEE Trans. Computers 48, 5 (1999), 506–521.
[19]
Aditya Menon, Omer Tamuz, Sumit Gulwani, Butler Lampson, and Adam Kalai. 2013. A machine learning framework for programming by example. In Proc. International Conference on Machine Learning. Proceedings of Machine Learning Research, 187–195.
[20]
Arvind Neelakantan, Quoc V Le, Martin Abadi, Andrew McCallum, and Dario Amodei. 2017. Learning a natural language interface with neural programmer. In Proc. International Conference on Learning Representations . OpenReview.
[21]
Arvind Neelakantan, Quoc V Le, and Ilya Sutskever. 2016. Neural programmer: Inducing latent programs with gradient descent. In Proc. International Conference on Learning Representations . OpenReview.
[22]
Peter-Michael Osera and Steve Zdancewic. 2015. Type-and-exampledirected program synthesis. In Proc. Conference on Programming Language Design and Implementation . ACM, 619–630.
[23]
Nadia Polikarpova, Ivan Kuraj, and Armando Solar-Lezama. 2016. Program synthesis from polymorphic refinement types. Proc. Conference on Programming Language Design and Implementation (2016), 522–538.
[24]
Veselin Raychev, Pavol Bielik, Martin Vechev, and Andreas Krause. 2016. Learning programs from noisy data. In Proc. Symposium on Principles of Programming Languages . ACM, 761–774.
[25]
Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code completion with statistical language models. In Proc. Conference on Programming Language Design and Implementation . ACM, 419–428.
[26]
Rishabh Singh. 2016. Blinkfill: Semi-supervised programming by example for syntactic string transformations. Proceedings of the VLDB Endowment 9, 10 (2016), 816–827.
[27]
Rishabh Singh and Sumit Gulwani. 2016. Transforming spreadsheet data types using examples. In Proc. Symposium on Principles of Programming Languages . ACM, 343–356.
[28]
Calvin Smith and Aws Albarghouthi. 2016. MapReduce program synthesis. In Proc. Conference on Programming Language Design and Implementation . ACM, 326–340.
[29]
Armando Solar-Lezama. 2008. Program synthesis by sketching. University of California, Berkeley.
[30]
Armando Solar-Lezama. 2009. The Sketching Approach to Program Synthesis. In Proc. Asian Symposium on Programming Languages and Systems . Springer, 4–13.
[31]
Armando Solar-Lezama, Rodric M. Rabbah, Rastislav Bodík, and Kemal Ebcioglu. 2005. Programming by sketching for bit-streaming programs. In Proc. Conference on Programming Language Design and Implementation . ACM, 281–294.
[32]
Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat. 2006. Combinatorial Sketching for Finite Programs. In Proc. International Conference on Architectural Support for Programming Languages and Operating Systems . ACM, 404–415.
[33]
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems . 3104–3112.
[34]
Ashish Tiwari, Adria Gascón, and Bruno Dutertre. 2015. Program synthesis using dual interpretation. In Proc. International Conference on Automated Deduction . Springer, 482–497.
[35]
Abhishek Udupa, Arun Raghavan, Jyotirmoy V Deshmukh, Sela MadorHaim, Milo MK Martin, and Rajeev Alur. 2013. TRANSIT: specifying protocols with concolic snippets. Proc. Conference on Programming Language Design and Implementation (2013), 287–296.
[36]
Chenglong Wang, Alvin Cheung, and Rastislav Bodik. 2017. Synthesizing highly expressive SQL queries from input-output examples. In Proc. Conference on Programming Language Design and Implementation . ACM, 452–466.
[37]
Xinyu Wang, Isil Dillig, and Rishabh Singh. 2017. Synthesis of Data Completion Scripts using Finite Tree Automata. In Proc. International Conference on Object-Oriented Programming, Systems, Languages, and Applications . ACM, 62:1–62:26.
[38]
Xinyu Wang, Isil Dillig, and Rishabh Singh. 2018. Program Synthesis using Abstraction Refinement. In Proc. Symposium on Principles of Programming Languages . ACM, 63:1–63:30.
[39]
Navid Yaghmazadeh, Christian Klinger, Isil Dillig, and Swarat Chaudhuri. 2016. Synthesizing transformations on hierarchically structured data. In Proc. Conference on Programming Language Design and Implementation . ACM, 508–521.
[40]
Navid Yaghmazadeh, Yuepeng Wang, Isil Dillig, and Thomas Dillig. 2017. SQLizer: Query Synthesis from Natural Language. In Proc. International Conference on Object-Oriented Programming, Systems, Languages, and Applications . ACM, 63:1–63:26.
[41]
Lintao Zhang, Conor F. Madigan, Matthew W. Moskewicz, and Sharad Malik. 2001. Efficient Conflict Driven Learning in Boolean Satisfiability Solver. In Proc. of International Conference on Computer-Aided Design. IEEE Computer Society, 279–285.
[42]
Sai Zhang and Yuyin Sun. 2013. Automatically synthesizing sql queries from input-output examples. In Proc. International Conference on Automated Software Engineering . IEEE, 224–234.

Cited By

View all
  • (2024)Recursive Program Synthesis using ParamorphismsProceedings of the ACM on Programming Languages10.1145/36563818:PLDI(102-125)Online publication date: 20-Jun-2024
  • (2024)Programming-by-Demonstration for Long-Horizon Robot TasksProceedings of the ACM on Programming Languages10.1145/36328608:POPL(512-545)Online publication date: 5-Jan-2024
  • (2024)Query Reverse Engineering of Pre-deleted Uncorrelated OperatorsData Mining and Big Data10.1007/978-981-97-0844-4_4(45-58)Online publication date: 22-Feb-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices
ACM SIGPLAN Notices  Volume 53, Issue 4
PLDI '18
April 2018
834 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3296979
Issue’s Table of Contents
  • cover image ACM Conferences
    PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation
    June 2018
    825 pages
    ISBN:9781450356985
    DOI:10.1145/3192366
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 June 2018
Published in SIGPLAN Volume 53, Issue 4

Check for updates

Author Tags

  1. automated reasoning
  2. conflict-driven learning
  3. program synthesis

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)390
  • Downloads (Last 6 weeks)63
Reflects downloads up to 12 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Recursive Program Synthesis using ParamorphismsProceedings of the ACM on Programming Languages10.1145/36563818:PLDI(102-125)Online publication date: 20-Jun-2024
  • (2024)Programming-by-Demonstration for Long-Horizon Robot TasksProceedings of the ACM on Programming Languages10.1145/36328608:POPL(512-545)Online publication date: 5-Jan-2024
  • (2024)Query Reverse Engineering of Pre-deleted Uncorrelated OperatorsData Mining and Big Data10.1007/978-981-97-0844-4_4(45-58)Online publication date: 22-Feb-2024
  • (2024)Relational Synthesis of Recursive Programs via Constraint Annotated Tree AutomataComputer Aided Verification10.1007/978-3-031-65633-0_3(41-63)Online publication date: 24-Jul-2024
  • (2024)Guiding Enumerative Program Synthesis with Large Language ModelsComputer Aided Verification10.1007/978-3-031-65630-9_15(280-301)Online publication date: 25-Jul-2024
  • (2023)Mobius: Synthesizing Relational Queries with Recursive and Invented PredicatesProceedings of the ACM on Programming Languages10.1145/36228477:OOPSLA2(1394-1417)Online publication date: 16-Oct-2023
  • (2023)Programming by Example Made EasyACM Transactions on Software Engineering and Methodology10.1145/360718533:1(1-36)Online publication date: 7-Jul-2023
  • (2023)Synthesizing MILP Constraints for Efficient and Robust OptimizationProceedings of the ACM on Programming Languages10.1145/35912987:PLDI(1896-1919)Online publication date: 6-Jun-2023
  • (2023)ImageEye: Batch Image Processing using Program SynthesisProceedings of the ACM on Programming Languages10.1145/35912487:PLDI(686-711)Online publication date: 6-Jun-2023
  • (2023)Unrealizability LogicProceedings of the ACM on Programming Languages10.1145/35712167:POPL(659-688)Online publication date: 11-Jan-2023
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media