Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3526113.3545619acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Notational Programming for Notebook Environments: A Case Study with Quantum Circuits

Published: 28 October 2022 Publication History

Abstract

We articulate a vision for computer programming that includes pen-based computing, a paradigm we term notational programming. Notational programming blurs contexts: certain typewritten variables can be referenced in handwritten notation and vice-versa. To illustrate this paradigm, we developed an extension, Notate, to computational notebooks which allows users to open drawing canvases within lines of code. As a case study, we explore quantum programming and designed a notation, Qaw, that extends quantum circuit notation with abstraction features, such as variable-sized wire bundles and recursion. Results from a usability study with novices suggest that users find our core interaction of implicit cross-context references intuitive, but suggests further improvements to debugging infrastructure, interface design, and recognition rates. Throughout, we discuss questions raised by the notational paradigm, including a shift from ‘recognition’ of notations to ‘reconfiguration’ of practices and values around programming, and from ‘sketching’ to writing and drawing, or what we call ‘notating.’

Supplementary Material

These are the Jupyter notebook task files used during our usability study for Notate. Included too are the "equivalent" Qiskit tasks used during our comparison step. (tasks.zip)
MP4 File (uist22-notate-video.mp4)
Video accompanying the original submission, depicting the interface.

References

[1]
Christine Alvarado and Randall Davis. 2007. SketchREAD: A Multi-Domain Sketch Recognition Engine. In ACM SIGGRAPH 2007 Courses (San Diego, California) (SIGGRAPH ’07). Association for Computing Machinery, New York, NY, USA, 34–es. https://doi.org/10.1145/1281500.1281545
[2]
Leif Andersen, Michael Ballantyne, and Matthias Felleisen. 2020. Adding Interactive Visual Syntax to Textual Code. Proc. ACM Program. Lang. 4, OOPSLA, Article 222 (Nov 2020), 28 pages. https://doi.org/10.1145/3428290
[3]
Georg Apitz and François Guimbretière. 2004. CrossY: A Crossing-Based Drawing Application. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (Santa Fe, NM, USA) (UIST ’04). Association for Computing Machinery, New York, NY, USA, 3–12. https://doi.org/10.1145/1029632.1029635
[4]
Ian Arawjo. 2020. To Write Code: The Cultural Fabrication of Programming Notation and Practice. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3313831.3376731
[5]
Zahra Ashktorab, Justin D. Weisz, and Maryam Ashoori. 2019. Thinking Too Classically: Research Topics in Human-Quantum Computer Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300486
[6]
Marat Boshernitsan and Michael Sean Downes. 2004. Visual programming languages: A survey. Citeseer.
[7]
Bill Buxton. 2010. Sketching user experiences: getting the design right and the right design. Morgan kaufmann.
[8]
Arnab Chakraborty. 2011. QuECT: a new quantum programming paradigm. arXiv preprint arXiv:1104.0497(2011).
[9]
Souti Chattopadhyay, Ishita Prasad, Austin Z. Henley, Anita Sarma, and Titus Barik. 2020. What’s Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376729
[10]
Michael Coblenz, Gauri Kambhatla, Paulette Koronkevich, Jenna L Wise, Celeste Barnaby, Joshua Sunshine, Jonathan Aldrich, and Brad A Myers. 2021. PLIERS: a process that integrates user-centered methods into programming language design. ACM Transactions on Computer-Human Interaction (TOCHI) 28, 4(2021), 1–53.
[11]
Michael Coblenz, Michelle L. Mazurek, and Michael Hicks. 2021. Does the Bronze Garbage Collector Make Rust Easier to Use? A Controlled Experiment. CoRR abs/2110.01098(2021). arXiv:2110.01098https://arxiv.org/abs/2110.01098
[12]
Bob Coecke and Aleks Kissinger. 2017. Picturing Quantum Processes. Cambridge University Press.
[13]
Randall Davis. 2007. Magic paper: Sketch-understanding research. Computer 40, 9 (2007), 34–41.
[14]
Marianne De Laet and Annemarie Mol. 2000. The Zimbabwe bush pump: Mechanics of a fluid technology. Social studies of science 30, 2 (2000), 225–263.
[15]
Gilles Deleuze and Félix Guattari. 1987. A thousand plateaus, trans. Brian Massumi.
[16]
Paul Dourish. 2017. The stuff of bits: An essay on the materialities of information. MIT Press.
[17]
Wenxiao Du. 2012. Code Runner: Solution for Recognition and Execution of Handwritten Code. Technical Report. Stanford University. 1–5 pages.
[18]
Anthony Dunne and Fiona Raby. 2001. Design noir: The secret life of electronic objects. Springer Science & Business Media.
[19]
Thomas O Ellis, John F Heafner, and William L Sibley. 1969. The GRAIL Project: An experiment in man-machine communications. Technical Report. RAND Corporation.
[20]
Martin Erwig and Bernd Meyer. 1995. Heterogeneous visual languages-integrating visual and textual programming. In Proceedings of Symposium on Visual Languages. IEEE, 318–325.
[21]
Craig Gidney. 2017. Quirk: Quantum Circuit Simulator. https://algassert.com/quirk.
[22]
Herman H. Goldstine and John Von Neumann. 1947. Planning and coding of problems for an electronic computing instrument. Technical Report. Moore School of Electrical Engineering, University of Pennsylvania.
[23]
Google. 2019. People + AI Guidebook. https://pair.withgoogle.com/guidebook/.
[24]
Google Quantum AI. 2022. Cirq. https://quantumai.google/cirq.
[25]
Alexander S. Green, Peter LeFanu Lumsdaine, Neil J. Ross, Peter Selinger, and Benoît Valiron. 2013. Quipper: A Scalable Quantum Programming Language. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (Seattle, Washington, USA) (PLDI ’13). Association for Computing Machinery, New York, NY, USA, 333–342. https://doi.org/10.1145/2491956.2462177
[26]
Thomas RG Green and Marian Petre. 1992. When visual programs are harder to read than textual programs. In Human-Computer Interaction: Tasks and Organisation, Proceedings ECCE-6 (6th European Conference Cognitive Ergonomics), Vol. 57. Citeseer.
[27]
Saul Greenberg and Bill Buxton. 2008. Usability Evaluation Considered Harmful (Some of the Time). In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI ’08). Association for Computing Machinery, New York, NY, USA, 111–120. https://doi.org/10.1145/1357054.1357074
[28]
Mark D. Gross and Ellen Yi-Luen Do. 1996. Ambiguous Intentions: A Paper-like Interface for Creative Design. In Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology (Seattle, Washington, USA) (UIST ’96). Association for Computing Machinery, New York, NY, USA, 183–192. https://doi.org/10.1145/237091.237119
[29]
Sumit Gulwani, Oleksandr Polozov, Rishabh Singh, 2017. Program synthesis. Foundations and Trends® in Programming Languages 4, 1-2(2017), 1–119.
[30]
Tracy Hammond and Randall Davis. 2007. LADDER, a Sketching Language for User Interface Developers. In ACM SIGGRAPH 2007 Courses (San Diego, California) (SIGGRAPH ’07). Association for Computing Machinery, New York, NY, USA, 35–es. https://doi.org/10.1145/1281500.1281546
[31]
Rebecca L. Hao and Elena L. Glassman. 2020. Approaching Polyglot Programming: What Can We Learn from Bilingualism Studies?. In 10th Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2019)(OpenAccess Series in Informatics (OASIcs), Vol. 76), Sarah Chasins, Elena L. Glassman, and Joshua Sunshine (Eds.). Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 1:1–1:7. https://doi.org/10.4230/OASIcs.PLATEAU.2019.1
[32]
Andrew Head, Fred Hohman, Titus Barik, Steven M. Drucker, and Robert DeLine. 2019. Managing Messes in Computational Notebooks. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300500
[33]
Brian Hempel, Justin Lubin, and Ravi Chugh. 2019. Sketch-n-Sketch: Output-Directed Programming for SVG. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 281–292. https://doi.org/10.1145/3332165.3347925
[34]
IBM. 2022. Qiskit Textbook: Learn Quantum Computation using Qiskit. https://qiskit.org/textbook/preface.html.
[35]
Gabe Johnson, Mark D Gross, Jason Hong, and Ellen Yi-Luen Do. 2008. Computational Support for Sketching in Design: A Review. Human–Computer Interaction 2, 1 (2008), 1–93.
[36]
David Kaiser. 2009. Drawing theories apart. University of Chicago Press.
[37]
DaYe Kang, Tony Ho, Nicolai Marquardt, Bilge Mutlu, and Andrea Bianchi. 2021. ToonNote: Improving Communication in Computational Notebooks Using Interactive Data Comics. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 727, 14 pages. https://doi.org/10.1145/3411764.3445434
[38]
Laewoo Kang and Steven Jackson. 2021. Tech-Art-Theory: Improvisational Methods for HCI Learning and Teaching. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 82 (Apr 2021), 25 pages. https://doi.org/10.1145/3449156
[39]
Mary Beth Kery, Donghao Ren, Fred Hohman, Dominik Moritz, Kanit Wongsuphasawat, and Kayur Patel. 2020. Mage: Fluid Moves Between Code and Graphical Work in Computational Notebooks. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, USA, 140–151. https://doi.org/10.1145/3379337.3415842
[40]
James A. Landay and Brad A. Myers. 1995. Interactive Sketching for the Early Stages of User Interface Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’95). ACM Press/Addison-Wesley Publishing Co., USA, 43–50. https://doi.org/10.1145/223904.223910
[41]
James A. Landay and Brad A. Myers. 2001. Sketching interfaces: toward more human interface design. Computer 34, 3 (2001), 56–64. https://doi.org/10.1109/2.910894
[42]
Chuanjun Li, Timothy S Miller, Robert C Zeleznik, and Joseph J LaViola Jr. 2008. AlgoSketch: Algorithm Sketching and Interactive Computation.SBIM 8(2008), 175–182.
[43]
Henry Lieberman. 2003. The Tyranny of Evaluation. ACM CHI Fringe (2003).
[44]
James Lin, Mark W. Newman, Jason I. Hong, and James A. Landay. 2000. DENIM: Finding a Tighter Fit between Tools and Practice for Web Site Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (The Hague, The Netherlands) (CHI ’00). Association for Computing Machinery, New York, NY, USA, 510–517. https://doi.org/10.1145/332040.332486
[45]
Microsoft. 2018. Sketch2Code. https://www.microsoft.com/en-us/ai/ai-lab-sketch2code.
[46]
Dhawal Mujumdar, Manuel Kallenbach, Brandon Liu, and Björn Hartmann. 2011. Crowdsourcing Suggestions to Programming Problems for Dynamic Web Development Languages. In CHI ’11 Extended Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI EA ’11). Association for Computing Machinery, New York, NY, USA, 1525–1530. https://doi.org/10.1145/1979742.1979802
[47]
Theresa O’Connell, Chuanjun Li, Timothy S. Miller, Robert C. Zeleznik, and Joseph J. LaViola. 2009. A Usability Evaluation of AlgoSketch: A Pen-Based Application for Mathematics. In Proceedings of the 6th Eurographics Symposium on Sketch-Based Interfaces and Modeling (New Orleans, Louisiana) (SBIM ’09). Association for Computing Machinery, New York, NY, USA, 149–157. https://doi.org/10.1145/1572741.1572767
[48]
Cyrus Omar and Jonathan Aldrich. 2018. Reasonably Programmable Literal Notation. Proc. ACM Program. Lang. 2, ICFP, Article 106 (Jul 2018), 32 pages. https://doi.org/10.1145/3236801
[49]
Cyrus Omar, David Moon, Andrew Blinn, Ian Voysey, Nick Collins, and Ravi Chugh. 2021. Filling Typed Holes with Live GUIs. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation (Virtual, Canada) (PLDI 2021). Association for Computing Machinery, New York, NY, USA, 511–525. https://doi.org/10.1145/3453483.3454059
[50]
Cyrus Omar, Ian Voysey, Ravi Chugh, and Matthew A. Hammer. 2019. Live Functional Programming with Typed Holes. Proc. ACM Program. Lang. 3, POPL, Article 14 (Jan 2019), 32 pages. https://doi.org/10.1145/3290327
[51]
Jennifer Paykin, Robert Rand, and Steve Zdancewic. 2017. QWIRE: A Core Language for Quantum Circuits. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (Paris, France) (POPL ’17). Association for Computing Machinery, New York, NY, USA, 846–858. https://doi.org/10.1145/3009837.3009894
[52]
Roger Penrose. 1971. Applications of negative dimensional tensors. Combinatorial mathematics and its applications 1 (1971), 221–244.
[53]
Mathys Rennela and Sam Staton. 2017. Classical Control, Quantum Circuits and Linear Logic in Enriched Category Theory. CoRR abs/1711.05159(2017). arXiv:1711.05159http://arxiv.org/abs/1711.05159
[54]
Adam Rule, Aurélien Tabard, and James D. Hollan. 2018. Exploration and Explanation in Computational Notebooks. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173606
[55]
Mark Santolucito. 2021. Human-in-the-Loop Program Synthesis for Live Coding. In Proceedings of the 9th ACM SIGPLAN International Workshop on Functional Art, Music, Modelling, and Design(Virtual, Republic of Korea) (FARM 2021). Association for Computing Machinery, New York, NY, USA, 47–53. https://doi.org/10.1145/3471872.3472972
[56]
Nazmus Saquib 2020. Embodied mathematics by interactive sketching. Ph. D. Dissertation. Massachusetts Institute of Technology.
[57]
Nazmus Saquib, Rubaiat Habib Kazi, Li-yi Wei, Gloria Mark, and Deb Roy. 2021. Constructing Embodied Algebra by Sketching. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 428, 16 pages. https://doi.org/10.1145/3411764.3445460
[58]
Lucy Suchman. 2002. Located accountabilities in technology production. Scandinavian journal of information systems 14, 2 (2002), 7.
[59]
Sarah Suleri, Vinoth Pandian Sermuga Pandian, Svetlana Shishkovets, and Matthias Jarke. 2019. Eve: A Sketch-Based Software Prototyping Workbench. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312994
[60]
Ivan E. Sutherland. 1963. Sketchpad, a Man-Machine Graphical Communication System. Ph. D. Dissertation. Massachusetts Institute of Technology, Cambridge, MA.
[61]
Krysta Svore, Alan Geller, Matthias Troyer, John Azariah, Christopher Granade, Bettina Heim, Vadym Kliuchnikov, Mariia Mykhailova, Andres Paz, and Martin Roetteler. 2018. Q# Enabling Scalable Quantum Computing and Development with a High-level DSL. In Proceedings of the Real World Domain Specific Languages Workshop 2018. 1–10.
[62]
Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 332, 7 pages. https://doi.org/10.1145/3491101.3519665
[63]
John Von Neumann. 1993. First Draft of a Report on the EDVAC. IEEE Annals of the History of Computing 15, 4 (1993), 27–75.
[64]
April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019. How Data Scientists Use Computational Notebooks for Real-Time Collaboration. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 39 (Nov 2019), 30 pages. https://doi.org/10.1145/3359141
[65]
April Yi Wang, Dakuo Wang, Jaimie Drozdal, Michael Muller, Soya Park, Justin D. Weisz, Xuye Liu, Lingfei Wu, and Casey Dugan. 2022. Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks. ACM Trans. Comput.-Hum. Interact. 29, 2, Article 17 (Jan 2022), 33 pages. https://doi.org/10.1145/3489465
[66]
Danli Wang, Yang Zhang, Tianyuan Gu, Liang He, and Hongan Wang. 2012. E-Block: A Tangible Programming Tool for Children. In Adjunct Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (Cambridge, Massachusetts, USA) (UIST Adjunct Proceedings ’12). Association for Computing Machinery, New York, NY, USA, 71–72. https://doi.org/10.1145/2380296.2380327
[67]
David Weintrop. 2019. Block-Based Programming in Computer Science Education. Commun. ACM 62, 8 (Jul 2019), 22–25. https://doi.org/10.1145/3341221
[68]
Kirsten N. Whitley. 1997. Visual programming languages and the empirical evidence for and against. Journal of Visual Languages & Computing 8, 1 (1997), 109–142.
[69]
Yifan Wu, Joseph M. Hellerstein, and Arvind Satyanarayan. 2020. B2: Bridging Code and Interactive Visualization in Computational Notebooks. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, USA, 152–165. https://doi.org/10.1145/3379337.3415851
[70]
Qian Yang, Aaron Steinfeld, Carolyn Rosé, and John Zimmerman. 2020. Re-Examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376301
[71]
Anna Zeng and Will Crichton. 2019. Identifying barriers to adoption for Rust through online discourse. arXiv preprint arXiv:1901.01001(2019).
[72]
Lei Zhang and Steve Oney. 2020. FlowMatic: An Immersive Authoring Tool for Creating Interactive Scenes in Virtual Reality. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, USA, 342–353. https://doi.org/10.1145/3379337.3415824
[73]
John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research through Design as a Method for Interaction Design Research in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 493–502. https://doi.org/10.1145/1240624.1240704

Cited By

View all
  • (2024)A Model-Driven Framework for Composition-Based Quantum Circuit DesignACM Transactions on Quantum Computing10.1145/36888565:4(1-36)Online publication date: 21-Aug-2024
  • (2024)Taking ASCII Drawings Seriously: How Programmers Diagram CodeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642683(1-16)Online publication date: 11-May-2024
  • (2024)Automated quantum software engineeringAutomated Software Engineering10.1007/s10515-024-00436-x31:1Online publication date: 12-Apr-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
UIST '22: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology
October 2022
1363 pages
ISBN:9781450393201
DOI:10.1145/3526113
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 October 2022

Permissions

Request permissions for this article.

Check for updates

Badges

  • Honorable Mention

Author Tags

  1. computational notebooks
  2. pen-based interfaces
  3. programming paradigms
  4. quantum computing

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

UIST '22

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25
The 38th Annual ACM Symposium on User Interface Software and Technology
September 28 - October 1, 2025
Busan , Republic of Korea

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)142
  • Downloads (Last 6 weeks)11
Reflects downloads up to 13 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Model-Driven Framework for Composition-Based Quantum Circuit DesignACM Transactions on Quantum Computing10.1145/36888565:4(1-36)Online publication date: 21-Aug-2024
  • (2024)Taking ASCII Drawings Seriously: How Programmers Diagram CodeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642683(1-16)Online publication date: 11-May-2024
  • (2024)Automated quantum software engineeringAutomated Software Engineering10.1007/s10515-024-00436-x31:1Online publication date: 12-Apr-2024
  • (2023)Lorgnette: Creating Malleable Code ProjectionsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606817(1-16)Online publication date: 29-Oct-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media