research-article

Open access

Small-Step Live Programming by Example

Authors:

Kasra Ferdowsifard,

Allen Ordookhanians,

Nadia PolikarpovaAuthors Info & Claims

UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

Pages 614 - 626

https://doi.org/10.1145/3379337.3415869

Published: 20 October 2020 Publication History

Abstract

Live programming is a paradigm in which the programming environment continually displays runtime values. Program synthesis is a technique that can generate programs or program snippets from examples. \deltextThis paper presents a new programming paradigm called Synthesis-Aided Live Programming that combines these two prior ideas in a synergistic way. When using Synthesis-Aided Live Programming, programmers can change the runtime values displayed by the live \addtextPrevious works that combine the two have taken a holistic approach to the way examples describe the behavior of functions and programs. This paper presents a new programming paradigm called Small-Step Live Programming by Example that lets the user apply Programming by Example locally. When using Small-Step Live Programming by Example, programmers can change the runtime values displayed by the live visualization to generate local program snippets. % Live programming and program % synthesis work perfectly together because the live programming environment % reifies values, which makes it easy for programmers to provide the examples % needed by the synthesizer. We implemented this new paradigm in a tool called \toolname, and performed a user study on $13$ programmers. Our study finds that Small-Step Live Programming by Example with \toolname helps users solve harder problems faster, and that for certain types of queries, users prefer it to searching the web. Additionally, we identify the \usersynthgap, in which users' mental models of the tool do not match its ability, and needs to be taken into account in the design of future synthesis tools.

Supplementary Material

VTT File (3379337.3415869.vtt)

Download
6.23 KB

MP4 File (ufp6963pv.mp4)

Preview video

Download
8.80 MB

MP4 File (ufp6963vf.mp4)

Video figure

Download
79.98 MB

MP4 File (3379337.3415869.mp4)

Presentation Video

Download
42.76 MB

References

[1]

2019. Alfie. https://alfie.prodo.ai/. (2019). Accessed: 2019-09-01.

[2]

2019. LightTable. http://lighttable.com/. (2019). Accessed: 2019-09-01.

[3]

Aws Albarghouthi, Sumit Gulwani, and Zachary Kincaid. 2013. Recursive program synthesis. In International Conference on Computer Aided Verification. Springer, 934--950.

[4]

Rajeev Alur, Dana Fisman, Rishabh Singh, and Armando Solar-Lezama. 2017. Sygus-comp 2017: Results and analysis. arXiv preprint arXiv:1711.11438 (2017).

[5]

Benjamin Biegel, Benedikt Lesch, and Stephan Diehl. 2015. Live object exploration: Observing and manipulating behavior and state of Java objects. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). 581--585. http://dx.doi.org/10.1109/ICSM.2015.7332518

Digital Library

[6]

Joel Brandt, Philip J. Guo, Joel Lewenstein, Mira Dontcheva, and Scott R. Klemmer. 2009. Two Studies of Opportunistic Programming: Interleaving Web Foraging, Learning, and Writing Code. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). Association for Computing Machinery, New York, NY, USA, 1589--1598. http://dx.doi.org/10.1145/1518701.1518944

[7]

Sarah E Chasins, Maria Mueller, and Rastislav Bodik. 2018. Rousillon: Scraping Distributed Hierarchical Web Data. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 963--975.

Digital Library

[8]

Ravi Chugh, Brian Hempel, Mitchell Spradlin, and Jacob Albers. 2016. Programmatic and Direct Manipulation, Together at Last. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '16). Association for Computing Machinery, New York, NY, USA, 341--354. http://dx.doi.org/10.1145/2908080.2908103

Digital Library

[9]

Ian Drosos, Titus Barik, Philip J. Guo, Robert DeLine, and Sumit Gulwani. 2020. Wrex: A Unified Programming-by-Example Interaction for Synthesizing Readable Code for Data Scientists. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. http://dx.doi.org/10.1145/3313831.3376442

Digital Library

[10]

Ethan Fast, Daniel Steffee, Lucy Wang, Joel R. Brandt, and Michael S. Bernstein. 2014. Emergent, Crowd-Scale Programming Practice in the IDE. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). Association for Computing Machinery, New York, NY, USA, 2491--2500. http://dx.doi.org/10.1145/2556288.2556998

[11]

Joel Galenson, Philip Reames, Rastislav Bodik, Björn Hartmann, and Koushik Sen. 2014. CodeHint: Dynamic and Interactive Synthesis of Code Snippets. In Proceedings of the 36th International Conference on Software Engineering (ICSE 2014). Association for Computing Machinery, New York, NY, USA, 653--663. http://dx.doi.org/10.1145/2568225.2568250

Digital Library

[12]

Katy Ilonka Gero, Zahra Ashktorab, Casey Dugan, Qian Pan, James Johnson, Werner Geyer, Maria Ruiz, Sarah Miller, David R Millen, Murray Campbell, and others. 2020. Mental Models of AI Agents in a Cooperative Game Setting. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--12.

Digital Library

[13]

Sumit Gulwani. 2011. Automating string processing in spreadsheets using input-output examples. ACM Sigplan Notices 46, 1 (2011), 317--330.

Digital Library

[14]

Sumit Gulwani, Mikaël Mayer, Filip Niksic, and Ruzica Piskac. 2015. StriSynth: synthesis for live programming. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 2. IEEE, 701--704.

[15]

Philip J. Guo. 2013. Online Python Tutor: Embeddable Web-based Program Visualization for Cs Education. In Proceeding of the 44th ACM Technical Symposium on Computer Science Education (SIGCSE '13). ACM, New York, NY, USA, 579--584. http://dx.doi.org/10.1145/2445196.2445368

Digital Library

[16]

Zheng Guo, Michael James, David Justo, Jiaxiao Zhou, Ziteng Wang, Ranjit Jhala, and Nadia Polikarpova. 2019. Program synthesis by type-guided abstraction refinement. Proceedings of the ACM on Programming Languages 4, POPL (2019), 1--28.

[17]

Tihomir Gvero, Viktor Kuncak, Ivan Kuraj, and Ruzica Piskac. 2013. Complete completion using types and weights. In ACM SIGPLAN Notices, Vol. 48. ACM, 27--38.

Digital Library

[18]

Christopher Michael Hancock. 2003. Real-time Programming and the Big Ideas of Computational Literacy. Ph.D. Dissertation. Cambridge, MA, USA. AAI0805688.

Digital Library

[19]

Brian Hempel, Justin Lubin, and Ravi Chugh. 2019. Sketch-n-Sketch: Output-Directed Programming for SVG. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 281--292.

Digital Library

[20]

Qinheping Hu, Roopsha Samanta, Rishabh Singh, and Loris D'Antoni. 2019. Direct Manipulation for Imperative Programs. In International Static Analysis Symposium. Springer, 347--367.

[21]

Jinru Hua and Sarfraz Khurshid. 2017. EdSketch: Execution-driven sketching for Java. In Proceedings of the 24th ACM SIGSOFT International SPIN Symposium on Model Checking of Software. ACM, 162--171.

Digital Library

[22]

Hyeonsu Kang and Philip J. Guo. 2017. Omnicode: A Novice-Oriented Live Programming Environment with Always-On Run-Time Value Visualizations. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, New York, NY, USA, 737--745. http://dx.doi.org/10.1145/3126594.3126632

Digital Library

[23]

Jan-Peter Kr"amer, Joachim Kurz, Thorsten Karrer, and Jan O. Borchers. 2014. How live coding affects developers' coding behavior. 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (2014), 5--8.

[24]

Tessa Lau and others. 2008. Why PBD systems fail: Lessons learned for usable AI. In CHI 2008 Workshop on Usable AI.

[25]

Vu Le and Sumit Gulwani. 2014. FlashExtract: a framework for data extraction by examples. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation. 542--553.

Digital Library

[26]

Xuan-Bach D Le, Duc-Hiep Chu, David Lo, Claire Le Goues, and Willem Visser. 2017. JFIX: semantics-based repair of Java programs via symbolic PathFinder. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. ACM, 376--379.

Digital Library

[27]

Sorin Lerner. 2020. Projection Boxes: On-the-Fly Reconfigurable Visualization for Live Programming. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/3313831.3376494

Digital Library

[28]

Hongwei Li, Zhao Xuejiao, Zhenchang Xing, Lingfeng Bao, Xin Peng, Dongjing Gao, and Wenyun Zhao. 2015. AmAssist: In-IDE ambient search of online programming resources. 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering, SANER 2015 - Proceedings (04 2015), 390--398. http://dx.doi.org/10.1109/SANER.2015.7081849

[29]

Fan Long and Martin Rinard. 2015. Staged program repair with condition synthesis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 166--178.

Digital Library

[30]

Fan Long and Martin Rinard. 2016. Automatic patch generation by learning correct code. ACM SIGPLAN Notices 51, 1 (2016), 298--312.

Digital Library

[31]

Mikaël Mayer, Viktor Kuncak, and Ravi Chugh. 2018. Bidirectional Evaluation with Direct Manipulation. Proc. ACM Program. Lang. 2, OOPSLA, Article Article 127 (Oct. 2018), 28 pages. http://dx.doi.org/10.1145/3276497

Digital Library

[32]

Cyrus Omar, Ian Voysey, Ravi Chugh, and Matthew A. Hammer. 2019. Live Functional Programming with Typed Holes. Proc. ACM Program. Lang. 3, POPL, Article 14 (Jan. 2019), 32 pages. http://dx.doi.org/10.1145/3290327

[33]

Antti Oulasvirta and Pertti Saariluoma. 2006. Surviving task interruptions: Investigating the implications of long-term working memory theory. International Journal of Human-Computer Studies 64, 10 (2006), 941--961.

Digital Library

[34]

Hila Peleg, Roi Gabay, Shachar Itzhaky, and Eran Yahav. 2020. Programming with a Read-Eval-Synth Loop. (2020). Manuscript submitted for publication.

[35]

Hila Peleg and Nadia Polikarpova. 2020. Perfect is the Enemy of Good: Best-Effort Program Synthesis. In 34th European Conference on Object-Oriented Programming, ECOOP 2020.

[36]

Hila Peleg, Sharon Shoham, and Eran Yahav. 2018. Programming not only by example. In Proceedings of the 40th International Conference on Software Engineering. ACM, 1114--1124.

Digital Library

[37]

David Rauch, Patrick Rein, Stefan Ramson, Jens Lincke, and Robert Hirschfeld. 2019. Babylonian-style Programming: Design and Implementation of an Integration of Live Examples Into General-purpose Source Code. The Art, Science, and Engineering of Programming 3, 3 (2019).

[38]

Mark Santolucito, William T Hallahan, and Ruzica Piskac. 2019. Live Programming By Example. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1--4.

[39]

Christopher Schuster and Cormac Flanagan. 2016. Live programming by example: using direct manipulation for live program synthesis. In LIVE Workshop.

[40]

Ben Shneiderman. 1983. Direct Manipulation: A Step Beyond Programming Languages. Computer 16, 8 (1983), 57--69.

Digital Library

[41]

Calvin Smith and Aws Albarghouthi. 2016. MapReduce program synthesis. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 326--340.

Digital Library

[42]

Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Sanjit Seshia, and Vijay Saraswat. 2006. Combinatorial sketching for finite programs. In Proceedings of the 12th international conference on Architectural support for programming languages and operating systems. 404--415.

Digital Library

[43]

S. L. Tanimoto. 2013. A perspective on the evolution of live programming. In 2013 1st International Workshop on Live Programming (LIVE). 31--34. http://dx.doi.org/10.1109/LIVE.2013.6617346

[44]

Abhishek Udupa, Arun Raghavan, Jyotirmoy V Deshmukh, Sela Mador-Haim, Milo MK Martin, and Rajeev Alur. 2013. TRANSIT: specifying protocols with concolic snippets. ACM SIGPLAN Notices 48, 6 (2013), 287--296.

Digital Library

[45]

Priyan Vaithilingam and Philip J Guo. 2019. Bespoke: Interactively Synthesizing Custom GUIs from Command-Line Applications By Demonstration. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 563--576.

Digital Library

[46]

Sahil Verma and Subhajit Roy. 2017. Synergistic Debug-Repair of Heap Manipulations. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2017). Association for Computing Machinery, New York, NY, USA, 163--173.

Digital Library

[47]

Bret Victor. 2012. Learnable Programming. (2012). http://worrydream.com/LearnableProgramming/

[48]

E. M. Wilcox, J. W. Atwood, M. M. Burnett, J. J. Cadiz, and C. R. Cook. 1997. Does Continuous Visual Feedback Aid Debugging in Direct-manipulation Programming Systems?. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI '97). ACM, New York, NY, USA, 258--265. http://dx.doi.org/10.1145/258549.258721

[49]

Glynn Winskel. 1993. The Formal Semantics of Programming Languages: An Introduction. MIT Press, Cambridge, MA, USA.

Digital Library

[50]

Y. Xiong, J. Wang, R. Yan, J. Zhang, S. Han, G. Huang, and L. Zhang. 2017. Precise Condition Synthesis for Program Repair. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), Vol. 00. 416--426. http://dx.doi.org/10.1109/ICSE.2017.45

[51]

Navid Yaghmazadeh, Xinyu Wang, and Isil Dillig. 2018. Automated Migration of Hierarchical Data to Relational Tables Using Programming-by-example. Proc. VLDB Endow. 11, 5 (Jan. 2018), 580--593. http://dx.doi.org/10.1145/3187009.3177735

Digital Library

Cited By

Mattis TKrebs ERinard MHirschfeld R(2024)Examples out of Thin Air: AI-Generated Dynamic Context to Assist Program Comprehension by ExampleCompanion Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming10.1145/3660829.3660845(99-107)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3660829.3660845
Tankelevitch LKewenig VSimkute AScott ASarkar ASellen ARintel S(2024)The Metacognitive Demands and Opportunities of Generative AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642902(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642902
Ferdowsi KHuang RJames MPolikarpova NLerner S(2024)Validating AI-Generated Code with Live ProgrammingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642495(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642495
Show More Cited By

Index Terms

Small-Step Live Programming by Example
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Graphical user interfaces
2. Software and its engineering
  1. Software creation and management
    1. Software development techniques
      1. Automatic programming

Recommendations

Live Programming By Example
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Live programming is a novel approach for programming practice. Programmers are given real-time feedback when writing code, traditionally via a graphical user interface. Despite live programming's practical values, such as providing an easier overview of ...
Live programming and programming by example: better together (keynote)
GPCE 2021: Proceedings of the 20th ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences

Live programming is a paradigm in which values from program execution are shown to the programmer through continual feedback. Programming by example is a paradigm in which code is synthesized from example values showing a desired behavior. This talk ...
Investigating the Impact of Using a Live Programming Environment in a CS1 Course
SIGCSE 2022: Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 1

Novice programmers often struggle with code understanding and debugging. Live Programming environments visualize the runtime values of a program each time it is modified to provide immediate feedback, which help with tracing the program execution. This ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

October 2020

1297 pages

ISBN:9781450375146

DOI:10.1145/3379337

General Chairs:
Shamsi Iqbal
Microsoft Research, USA
,
Karon MacLean
University of British Columbia, Canada
,
Program Chairs:
Fanny Chevalier
University of Toronto, Canada
,
Stefanie Mueller
MIT CSAIL, USA

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

UIST '20

Sponsor:

UIST '20: The 33rd Annual ACM Symposium on User Interface Software and Technology

October 20 - 23, 2020

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
1,481
Total Downloads

Downloads (Last 12 months)463
Downloads (Last 6 weeks)44

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mattis TKrebs ERinard MHirschfeld R(2024)Examples out of Thin Air: AI-Generated Dynamic Context to Assist Program Comprehension by ExampleCompanion Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming10.1145/3660829.3660845(99-107)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3660829.3660845
Tankelevitch LKewenig VSimkute AScott ASarkar ASellen ARintel S(2024)The Metacognitive Demands and Opportunities of Generative AIProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642902(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642902
Ferdowsi KHuang RJames MPolikarpova NLerner S(2024)Validating AI-Generated Code with Live ProgrammingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642495(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642495
Liang JYang CMyers BRoychoudhury APaiva AAbreu RStorey M(2024)A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and ChallengesProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3608128(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3608128
Abdollahi FAmeen STaylor MLelis LElkind E(2023)Can you improve my code? optimizing programs with local searchProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/328(2940-2948)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/328
Nazari AHuang YSamanta RRadhakrishna ARaghothaman M(2023)Explainable Program Synthesis by Localizing SpecificationsProceedings of the ACM on Programming Languages10.1145/36228747:OOPSLA2(2171-2195)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3622874
Liu XShi YYu CGao CYang TLiang CShi Y(2023)Understanding In-Situ Programming for Smart Home AutomationProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962547:2(1-31)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3596254
Horowitz JHeer J(2023)Engraft: An API for Live, Rich, and Composable ProgrammingProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606733(1-18)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606733
Barke SJames MPolikarpova N(2023)Grounded Copilot: How Programmers Interact with Code-Generating ModelsProceedings of the ACM on Programming Languages10.1145/35860307:OOPSLA1(85-111)Online publication date: 6-Apr-2023
https://dl.acm.org/doi/10.1145/3586030
Mcnutt AWang CDeline RDrucker S(2023)On the Design of AI-powered Code Assistants for NotebooksProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580940(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580940
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents