research-article

Open access

Adding interactive visual syntax to textual code

Authors:

Michael Ballantyne,

Matthias FelleisenAuthors Info & Claims

Proceedings of the ACM on Programming Languages, Volume 4, Issue OOPSLA

Article No.: 222, Pages 1 - 28

https://doi.org/10.1145/3428290

Published: 13 November 2020 Publication History

Abstract

Many programming problems call for turning geometrical thoughts into code: tables, hierarchical structures, nests of objects, trees, forests, graphs, and so on. Linear text does not do justice to such thoughts. But, it has been the dominant programming medium for the past and will remain so for the foreseeable future.

This paper proposes a novel mechanism for conveniently extending textual programming languages with problem-specific visual syntax. It argues the necessity of this language feature, demonstrates the feasibility with a robust prototype, and sketches a design plan for adapting the idea to other languages.

Supplementary Material

Auxiliary Presentation Video (oopsla20main-p510-p-video.mp4)

Many programming problems call for turning geometrical thoughts into code: tables, hierarchical structures, nests of objects, trees, forests, graphs, and so on. Linear text does not do justice to such thoughts. But, it has been the dominant programming medium for the past and will remain so for the foreseeable future. This paper proposes a novel mechanism for conveniently extending textual programming languages with problem-specific visual syntax. It argues the necessity of this language feature, demonstrates the feasibility with a robust prototype, and sketches a design plan for adapting the idea to other languages.

Download
55.59 MB

References

[1]

Adobe. Adobe Dreamweaver CC Help. Retrieved May, 2020, 2019. https://helpx.adobe.com/pdf/dreamweaver_reference.pdf

[2]

Leif Andersen, Stephen Chang, and Matthias Felleisen. Super 8 Languages for Making Movies (Functional Pearl). Proceedings of the ACM on Programming Languages 1(International Conference on Functional Programming), pp. 30-1-30-29, 2017. https://doi.org/10.1145/3110274

Digital Library

[3]

Jeremy Ashkenas. Observable: The User Manual. Retrieved February, 2020, 2019. https://observablehq.com/ @observablehq/user-manual

[4]

Rudolf Bayer. Symmetric Binary B-Trees : Data Structure and Maintenance Algorithms. Acta Informatica 1 ( 4 ), pp. 290-306, 1972. https://doi.org/10.1007/BF00289509

Digital Library

[5]

Alexandre Bergel, Damien Cassou, Stéphane Ducasse, and Jannik Laval. Deep into Pharo. Square Bracket Associates, 2013.

[6]

L. Bernardin, P. Chin, P. DeMarco, K. O. Geddes, D. E. G. Hare, K. M. Heal, G. Labahn, J. P. May, J. McCarron, M. B. Monagan, D. Ohashi, and S. M. Vorkoetter. Maple Programming Guide. Maplesoft, 2012.

[7]

Marat Boshernitsan and Michael S. Downes. Visual Programming Languages: a Survey. EECS Department, University of California, Berkeley, UCB/CSD-04-1368, 2004. http://www2.eecs.berkeley.edu/Pubs/TechRpts/ 2004/6201.html

[8]

Ravi Chugh, Brian Hempel, Mitchell Spradlin, and Jacob Albers. Programmatic and Direct Manipulation, Together at Last. In Proc. Programming Languages Design and Implementation, pp. 341-354, 2016. https: //doi.org/10.1145/2980983.2908103

Digital Library

[9]

Hubert Comon, Max Dauchet, Remi Gilleron, Florent Jacquemard, Denis Lugiez, Christof Löding, Sophie Tison, and Marc Tommasi. Tree Automata Techniques and Applications. 2007. http://tata.gforge.inria.fr/

[10]

Gregory Cooper and Shriram Krishnamurthi. Embedding Dynamic Dataflow in a Call-by-Value Language. In Proc. European Symposium on Programming, pp. 294-308, 2004. https://doi.org/10.1007/11693024_20

Digital Library

[11]

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Cliford Stein. Introduction to Algorithms, Third Edition. MIT Press, 2009.

Digital Library

[12]

Andrea A. diSessa and Harold Abelson. Boxer: A Reconstructible Computational Medium. Communications of the ACM 29 ( 9 ), pp. 859-868, 1986. https://doi.org/10.1145/6592.6595

Digital Library

[13]

R. Kent Dybvig. The Development of Chez Scheme. In Proc. International Conference on Functional Programming, pp. 1-12, 2006. https://doi.org/10.1145/1160074.1159805

Digital Library

[14]

Andrew D. Eisenberg and Gregor Kiczales. Expressive Programs Through Presentation Extension. In Proc. International Conference on Aspect-Oriented Software Development, pp. 73-84, 2007. https://doi.org/10. 1145/1218563.1218573

Digital Library

[15]

T. O. Ellis, J. F. Heafner, and W. L. Sibley. The GRAIL Language and Operations. RAND Corporation, RM-6001-ARPA, 1969a. https://doi.org/10.7249/RM6001

[16]

T. O. Ellis, J. F. Heafner, and W. L. Sibley. The Grail Project: An Experiment in Man-Machine Communications. RAND Corporation, RM-5999-ARPA, 1969b. https://www.rand.org/pubs/research_memoranda/RM5999. html

[17]

Sebastian Erdweg, Lennart C. L. Kats, Tillmann Rendel, Christian Kästner, Klaus Ostermann, and Eelco Visser. Growing a Language Environment with Editor Libraries. In Proc. Generative Programming and Component Engineering, pp. 167-176, 2011. https://doi.org/10.1145/2189751.2047891

Digital Library

[18]

Matthias Felleisen, Robert Bruce Findler, Matthew Flatt, Shriram Krishnamurthi, Eli Barzilay, Jay McCarthy, and Sam Tobin-Hochstadt. A Programmable Programming Language. Communications of the ACM 61 ( 3 ), pp. 62-71, 2018. https://doi.org/10.1145/3127323

Digital Library

[19]

Robert Bruce Findler, John Clements, Cormac Flanagan, Matthew Flatt, Shriram Krishnamurthi, Paul Steckler, and Matthias Felleisen. DrScheme: A Programming Environment for Scheme. Journal of Functional Programming 12 ( 2 ), pp. 159-182, 2002. https://doi.org/10.1017/S0956796801004208

Digital Library

[20]

Robert Bruce Findler and PLT. DrRacket: Programming Environment. PLT Design Inc., PLT-TR-2010-2, 2010. https://racket-lang. org/tr2/

[21]

Matthew Flatt. Composable and Compilable Macros, You Want It When? In Proc. International Conference on Functional Programming, pp. 72-83, 2002.

[22]

Matthew Flatt. Submodules in Racket, You Want it When, Again? In Proc. Generative Programming: Concepts & Experiences, pp. 13-22, 2013. https://doi.org/10.1145/2517208.2517211

Digital Library

[23]

Matthew Flatt, Robert Bruce Findler, and John Clements. GUI : Racket Graphics Toolkit. PLT Design Inc., PLT-TR-2010-3, 2010. https://racket-lang. org/tr3/

[24]

Matthew Flatt, Robert Bruce Findler, and Matthias Felleisen. Scheme with Classes, Mixins, and Traits. In Proc. Asian Symposium Programming Languages and Systems, pp. 270-289, 2006.

Digital Library

[25]

Matthew Flatt and PLT. Reference: Racket. PLT Design Inc., PLT-TR-2010-1, 2010. https://racket-lang. org/tr1/

[26]

Robert Fourer, David M. Gay, and Brian W. Kernighan. AMPL: A Modeling Language for Mathematical Programming. 2nd edition. Cengage Learning, 2002. https://ampl.com/resources/the-ampl-book/

[27]

G. W. French, J. R. Kennaway, and A. M. Day. Programs as Visual, Interactive Documents. Journal of Software: Practice and Experience 44 ( 8 ), pp. 911-930, 2014. https://doi.org/10.1002/spe.2182

Digital Library

[28]

Adele Goldberg and David Robson. Smalltalk-80 : The Language and Its Implementation. Addison-Wesley Longman Publishing Co, 1983.

Digital Library

[29]

Brian Harvey and Jens Mönig. Bringing "No Ceiling" to Scratch: Can One Language Serve Kids and Computer Scientists? In Proc. Constructionism, pp. 1-10, 2010.

[30]

Brian Hempel, Justin Lubin, Grace Lu, and Ravi Chugh. Deuce: A Lightweight User Interface for Structured Editing. In Proc. International Conference on Software Engineering, pp. 654-664, 2018. https://doi.org/10. 1145/3180155.3180165

Digital Library

[31]

Daniel Ingalls, Krzysztof Palacz, Stephen Uhler, Antero Taivalsaari, and Tommi Mikkonen. The Lively Kernel A Self-supporting System on a Web Page. In Proc. Self-Sustaining Systems, pp. 31-50, 2008. https://doi. org/10.1007/978-3-540-89275-5_2

Digital Library

[32]

Lennart C. L. Kats and Eelco Visser. The Spoofax Language Workbench. In Proc. Object-Oriented Programming, Systems, Languages & Applications, pp. 444-463, 2010. https://doi.org/10.1145/1932682.1869497

Digital Library

[33]

Clemens N. Klokmose, James R. Eagan, Siemen Baader, Wendy Mackay, and Michel Beaudouin-Lafon. Webstrates: Shareable Dynamic Media. In Proc. ACM Symposium on User Interface Software and Technology, pp. 280-290, 2015. https://doi.org/10.1145/2807442.2807446

Digital Library

[34]

Amy Ko and Brad A. Myers. Barista: An Implementation Framework for Enabling New Tools, Interaction Techniques and Views in Code Editors. In Proc. Conference on Human Factors in Computing Systems, pp. 387-396, 2006. https://doi.org/10.1145/1124772.1124831

Digital Library

[35]

John Maloney, Kimberly M. Rose, and Walt Disney Imagineering. An Introduction to Morphic: The Squeak User Interface Framework. In Squeak: Open Personal Computing and Multimedia, pp. 39-77 Pearson, 2001.

[36]

Microsoft. Ofice and SharePoint Development in Visual Studio. Retrieved January, 2019, 2019. https://docs.microsoft.com/en-us/visualstudio/vsto/ofice-and-sharepoint-development-in-visualstudio?view=vs-2017

[37]

Chris Okasaki. Red-black Trees in a Functional Setting. Journal of Functional Programming 9 ( 4 ), pp. 471-477, 1999. https://doi.org/10.1017/S0956796899003494

Digital Library

[38]

Cyrus Omar, Nick Collins, David Moon, Ian Voysey, and Ravi Chugh. Livelits: Filling Typed Holes with Live GUIs (Extended Abstract). In Proc. Workshop on Type-driven Development, 2019.

[39]

Cyrus Omar, YoungSeok Yoon, Thomas D. LaToza, and Brad A. Myers. Active Code Completion. In Proc. International Conference on Software Engineering, pp. 859-869, 2012.

[40]

Mark Overmars. Teaching Computer Science Through Game Design. Computer 37 ( 4 ), pp. 81-83, 2004. https: //doi.org/10.1109/ MC. 2004.1297314

Digital Library

[41]

Vaclav Pech, Alex Shatalin, and Markus Voelter. JetBrains MPS as a Tool for Extending Java. In Proc. Principles and Practice of Programming in Java, pp. 165-168, 2013. https://doi.org/10.1145/2500828.2500846

Digital Library

[42]

Fernando Perez and Brian E. Granger. IPython: A System for Interactive Scientific Computing. Computing in Science and Engineering 9 ( 3 ), pp. 21-29, 2007. https://doi.org/10.1109/ MCSE. 2007.53

Digital Library

[43]

Jon Postel. Transmission Control Protocol. Internet Engineering Task Force, RFC 793, 1981. https://tools.ietf. org/html/rfc793

[44]

Mitchel Resnick, John Maloney, Andrés Monroy-Hernández, Natalie Rusk, Evelyn Eastmond, Karen Brennan, Amon Millner, Eric Rosenbaum, Jay Silver, Brian Silverman, and Yasmin Kafai. Scratch: Programming for All. Communications of the ACM 52 ( 11 ), pp. 60-67, 2009. https://doi.org/10.1145/1592761.1592779

Digital Library

[45]

Roman Rädle, Midas Nouwens, Kristian Antonsen, James R. Eagan, and Clemens N. Klokmose. Codestrates: Literate Computing with Webstrates. In Proc. ACM Symposium on User Interface Software and Technology, pp. 715-725, 2017. https://doi.org/10.1145/3126594.3126642

Digital Library

[46]

Charles Simonyi, Magnus Christerson, and Shane Cliford. Intentional Software. ACM SIGPLAN Notices 41 ( 10 ), pp. 451-464, 2006. https://doi.org/10.1145/1167515.1167511

Digital Library

[47]

Markus Voelter and Sascha Lisson. Supporting Diverse Notations in MPS' Projectional Editor. In Proc. International Workshop on The Globalization of Modeling Languages, 2014.

[48]

Markus Voelter, Daniel Ratiu, Bernhard Schaetz, and Bernd Kolb. mbeddr: an Extensible C-based Programming Language and IDE for Embedded Systems. In Proc. Conference on Systems, Programming, and Applications: Software for Humanity, pp. 121-140, 2012. https://doi.org/10.1145/2384716.2384767

Digital Library

[49]

Holger Vogt, Marcel Hendrix, and Paolo Nenzi. Ngspice Users Manual. NGSPICE, 30, 2019. http://ngspice. sourceforge.net/docs/ngspice-30-manual.pdf

Cited By

Bohrer R(2024)Programming Language Case Studies Can Be DeepElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.405.4405(56-79)Online publication date: 10-Jul-2024
https://doi.org/10.4204/EPTCS.405.4
Nelson TGreenman BPrasad SDyer TBove EChen QCutting CDel Vecchio TLeVine SRudner JRyjikov BVarga AWagner AWest LKrishnamurthi S(2024)Forge: A Tool and Language for Teaching Formal MethodsProceedings of the ACM on Programming Languages10.1145/36498338:OOPSLA1(613-641)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649833
Hayatpur DHempel BChen KDuan WGuo PXia H(2024)Taking ASCII Drawings Seriously: How Programmers Diagram CodeProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642683(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642683
Show More Cited By

Index Terms

Adding interactive visual syntax to textual code
1. Human-centered computing
  1. Accessibility
    1. Accessibility technologies
  2. Human computer interaction (HCI)
2. Software and its engineering
  1. Software notations and tools
    1. Context specific languages
    2. Development frameworks and environments
      1. Integrated and visual development environments

Recommendations

The Domain-Specific Language Monaco and its Visual Interactive Programming Environment
VLHCC '07: Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing

Monaco is a domain-specific language for machine automation programming. It has been developed with the objective to empower domain experts with limited programming capabilities. Its main language features are an imperative notation for reactive systems,...
Compiling Embedded Programs to Byte Code
PADL '02: Proceedings of the 4th International Symposium on Practical Aspects of Declarative Languages

Functional languages have proven substantially useful for hosting embedded domain-specific languages. They provide an infrastructure rich enough to define both a convenient syntax for the embedded language, a type system for embedded programs, and an ...
JGroovy - an extensible Java programming language with Groovy
ICACT'10: Proceedings of the 12th international conference on Advanced communication technology

Java is Object-Oriented Programming Languages (OOPL) that widely used for software development. But Java has a limitation for working with Domain-Specific Languages (DSLs). Java language structure and syntax has not more support for working with DSLs ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages

Proceedings of the ACM on Programming Languages Volume 4, Issue OOPSLA

November 2020

3108 pages

EISSN:2475-1421

DOI:10.1145/3436718

Issue’s Table of Contents

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 November 2020

Published in PACMPL Volume 4, Issue OOPSLA

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tag

Domain-Specific Language

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
1,427
Total Downloads

Downloads (Last 12 months)321
Downloads (Last 6 weeks)22

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bohrer R(2024)Programming Language Case Studies Can Be DeepElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.405.4405(56-79)Online publication date: 10-Jul-2024
https://doi.org/10.4204/EPTCS.405.4
Nelson TGreenman BPrasad SDyer TBove EChen QCutting CDel Vecchio TLeVine SRudner JRyjikov BVarga AWagner AWest LKrishnamurthi S(2024)Forge: A Tool and Language for Teaching Formal MethodsProceedings of the ACM on Programming Languages10.1145/36498338:OOPSLA1(613-641)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649833
Hayatpur DHempel BChen KDuan WGuo PXia H(2024)Taking ASCII Drawings Seriously: How Programmers Diagram CodeProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642683(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642683
Korz NAndrzejak ADe Roover CRumpe BShaikhha A(2023)Virtual Domain Specific Languages via Embedded Projectional EditingProceedings of the 22nd ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/3624007.3624059(122-137)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3624007.3624059
Beckmann TStachnik DLincke JHirschfeld RBeckmann THirschfeld RSáenz JMerino M(2023)Visual Replacements: Cross-Language Domain-Specific Representations in Structured EditorsProceedings of the 2nd ACM SIGPLAN International Workshop on Programming Abstractions and Interactive Notations, Tools, and Environments10.1145/3623504.3623569(25-35)Online publication date: 18-Oct-2023
https://dl.acm.org/doi/10.1145/3623504.3623569
Kavalar MMarkovics PRusher J(2023)Clerk: Moldable Live Programming for ClojureCompanion Proceedings of the 7th International Conference on the Art, Science, and Engineering of Programming10.1145/3594671.3594682(22-31)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3594671.3594682
Hirzel M(2023)Low-Code Programming ModelsCommunications of the ACM10.1145/358769166:10(76-85)Online publication date: 22-Sep-2023
https://dl.acm.org/doi/10.1145/3587691
Gobert CBeaudouin-Lafon M(2023)Lorgnette: Creating Malleable Code ProjectionsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606817(1-16)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606817
Horowitz JHeer J(2023)Engraft: An API for Live, Rich, and Composable ProgrammingProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606733(1-18)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606733
Mcnutt AOutkine AChugh R(2023)A Study of Editor Features in a Creative Coding ClassroomProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580683(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580683
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents