research-article

Open access

One tool, many languages: language-parametric transformation with incremental parametric syntax

Authors:

Varot Premtoon,

Armando Solar-LezamaAuthors Info & Claims

Proceedings of the ACM on Programming Languages, Volume 2, Issue OOPSLA

Article No.: 122, Pages 1 - 28

https://doi.org/10.1145/3276492

Published: 24 October 2018 Publication History

Abstract

We present a new approach for building source-to-source transformations that can run on multiple programming languages, based on a new way of representing programs called incremental parametric syntax. We implement this approach in Haskell in our Cubix system, and construct incremental parametric syntaxes for C, Java, JavaScript, Lua, and Python. We demonstrate a whole-program refactoring tool that runs on all of them, along with three smaller transformations that each run on several. Our evaluation shows that (1) once a transformation is written, little work is required to configure it for a new language (2) transformations built this way output readable code which preserve the structure of the original, according to participants in our human study, and (3) our transformations can still handle language corner-cases, as validated on compiler test suites.

Supplementary Material

WEBM File (a122-koppel.webm)

Download
104.84 MB

References

[1]

Franz Baader and Tobias Nipkow. 1999. Term Rewriting and All That. Cambridge university press.

Digital Library

[2]

Patrick Bahr and Tom Hvitved. 2011. Compositional Data Types. In Proceedings of the Seventh ACM SIGPLAN Workshop on Generic programming, WGP@ICFP 2011, Tokyo, Japan, September 19-21, 2011. 83–94.

Digital Library

[3]

Ira D Baxter, Christopher Pidgeon, and Michael Mehlich. 2004. DMS®: Program Transformations for Practical Scalable Software Evolution. In Proceedings of the 26th International Conference on Software Engineering. IEEE Computer Society, 625–634.

Digital Library

[4]

Denis Bogdanas and Grigore Roşu. 2015. K-Java: A Complete Semantics of Java. In Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2015, Mumbai, India, January 15-17, 2015. 445–456.

Digital Library

[5]

Niklas Broberg. 2015. language-java: Manipulating Java source: abstract syntax, lexer, parser, and pretty-printer. http: //hackage.haskell.org/package/language-java-0.2.8 . (November 2015).

[6]

Fraser Brown, Andres Nötzli, and Dawson Engler. 2016. How to Build Static Checking Systems Using Orders of Magnitude Less Code. In Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, 143–157.

Digital Library

[7]

David Brumley, Ivan Jager, Thanassis Avgerinos, and Edward J. Schwartz. 2011. BAP: A Binary Analysis Platform. In CAV.

Digital Library

[8]

Martin Churchill, Peter D Mosses, Neil Sculthorpe, and Paolo Torrini. 2015. Reusable Components of Semantic Specifications. In Transactions on Aspect-Oriented Software Development XII. Springer, 132–179.

[9]

Manuel Clavel, Francisco Durán, Steven Eker, Patrick Lincoln, Narciso Martı-Oliet, José Meseguer, and José F Quesada. 2002. Maude: Specification and Programming in Rewriting Logic. Theoretical Computer Science 285, 2 (2002), 187–243.

Digital Library

[10]

James R Cordy. 2006. The TXL Source Transformation Language. Science of Computer Programming 61, 3 (2006), 190–210.

Digital Library

[11]

Bruno C. d. S. Oliveira, Shin-Cheng Mu, and Shu-Hung You. 2015. Modular Reifiable Matching: A List-of-Functors Approach to Two-level Types. In Proceedings of the 8th ACM SIGPLAN Symposium on Haskell, Haskell 2015, Vancouver, BC, Canada, September 3-4, 2015. 82–93.

Digital Library

[12]

Benjamin Delaware, Steven Keuchel, Tom Schrijvers, and Bruno C. d. S. Oliveira. 2013. Modular Monadic Meta-Theory. In ACM SIGPLAN International Conference on Functional Programming, ICFP’13, Boston, MA, USA - September 25 - 27, 2013. 319–330.

Digital Library

[13]

Isil Dillig, Thomas Dillig, and Alex Aiken. 2009. SAIL: Static Analysis Intermediate Language with a Two-level Representation. Stanford University Technical Report (2009).

[14]

Tom Dinkelaker, Michael Eichberg, and Mira Mezini. 2013. Incremental Concrete Syntax for Embedded Languages with Support for Separate Compilation. Science of Computer Programming 78, 6 (2013), 615–632.

Digital Library

[15]

Michael Dory, Allison Parrish, and Brendan Berg. 2012. Introduction to Tornado: Modern Web Applications with Python. O’Reilly Media, Inc.

Digital Library

[16]

Sebastian Erdweg, Tijs van der Storm, Markus Völter, Meinte Boersma, Remi Bosman, William R Cook, Albert Gerritsen, Angelo Hulshout, Steven Kelly, Alex Loh, et al. 2013. The State of the Art in Language Workbenches. In International Conference on Software Language Engineering. Springer, 197–217.

[17]

Python Software Foundation. 2016. CPython Test Suite. Version 3.7.0a0. https://docs.python.org/devguide/runtests.html . (October 2016).

[18]

FSF. 2016. C Language Testsuites: “C-torture”. Revision 240758. http://gcc.gnu.org/onlinedocs/gccint/C-Tests.html . (October 2016).

[19]

Andy Gill. 2009. A Haskell Hosted DSL for Writing Transformation Systems. In Domain-Specific Languages. Springer, 285–309.

Digital Library

[20]

Matthias Grimmer, Chris Seaton, Roland Schatz, Thomas Würthinger, and Hanspeter Mössenböck. 2015. High-performance cross-language interoperability in a multi-language runtime. In Proceedings of the 11th Symposium on Dynamic Languages, DLS 2015, part of SPLASH 2015, Pittsburgh, PA, USA, October 25-30, 2015. 78–90.

Digital Library

[21]

Miguel Grinberg. 2014. Flask Web Development: Developing Web Applications with Python. O’Reilly Media, Inc.

Digital Library

[22]

Jan Heering and Ralf Lämmel. 2004. Generic Software Transformations. In Proceedings of the Software Transformation Systems Workshop.

[23]

Benedikt Huber. 2016. language-c: Analysis and generation of C code. http://hackage.haskell.org/package/language-c . (2016).

[24]

JDK Bug System. 2016. javac crash when local from enclosing context is captured multiple times. https://bugs.openjdk.java. net/browse/JDK-8169345 . (2016).

[25]

Lennart CL Kats and Eelco Visser. 2010. The Spoofax Language Workbench: Rules for Declarative Specification of Languages and IDEs. Vol. 45. ACM.

[26]

Oleg Kiselyov. 2012. Typed Tagless Final Interpreters. In Generic and Indexed Programming. Springer, 130–174.

Digital Library

[27]

Paul Klint, Tijs van der Storm, and Jurgen Vinju. 2009. EASY Meta-programming with Rascal. In International Summer School on Generative and Transformational Techniques in Software Engineering. Springer, 222–289.

Digital Library

[28]

James Koppel, Varot Premtoon, and Armando Solar-Lezama. 2018. One Tool, Many Languages: Language-Parametric Transformation with Incremental Parametric Syntax. CoRR abs/1707.04600 (2018). arXiv: 1707.04600 http://arxiv.org/abs/ 1707.04600

Digital Library

[29]

James Koppel and Armando Solar-Lezama. 2017. Incremental parametric syntax for multi-language transformation. In Proceedings Companion of the 2017 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity, SPLASH 2017, Vancouver, BC, Canada, October 23 - 27, 2017. 53–54.

Digital Library

[30]

Ralf Lämmel. 2002. Towards Generic Refactoring. In Proceedings of the 2002 ACM SIGPLAN workshop on Rule-based programming. ACM, 15–28.

Digital Library

[31]

Ralf Lämmel and Simon Peyton Jones. 2003. Scrap Your Boilerplate: A Practical Design Pattern for Generic Programming. Vol. 38. ACM.

[32]

Ralf Lämmel, Eelco Visser, and Joost Visser. 2003. Strategic Programming Meets Adaptive Programming. In Proceedings of the 2Nd International Conference on Aspect-oriented Software Development (AOSD ’03). ACM, New York, NY, USA, 168–177.

Digital Library

[33]

Ralf Lämmel and Joost Visser. 2002. Typed Combinators for Generic Traversal. In International Symposium on Practical Aspects of Declarative Languages. Springer, 137–154.

Digital Library

[34]

Ralf Lämmel and Joost Visser. 2003. A Strafunski Application Letter. In International Symposium on Practical Aspects of Declarative Languages. Springer, 357–375.

Digital Library

[35]

Chris Lattner and Vikram S. Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In 2nd IEEE / ACM International Symposium on Code Generation and Optimization (CGO 2004), 20-24 March 2004, San Jose, CA, USA. 75–88.

Digital Library

[36]

Sheng Liang, Paul Hudak, and Mark Jones. 1995. Monad Transformers and Modular Interpreters. In Proceedings of the 22nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 333–343.

Digital Library

[37]

Tim Lindholm, Frank Yellin, Gilad Bracha, and Alex Buckley. 2014. The Java Virtual Machine Specification: Java SE 8 Edition. Pearson Education.

Digital Library

[38]

Sam Madden. 2017. 6.830 Lab 1: SimpleDB. (2017). http://db.csail.mit.edu/6.830/assignments/lab1.html

[39]

Leon Moonen. 2001. Generating Robust Parsers using Island Grammars. In Reverse Engineering, 2001. Proceedings. Eighth Working Conference on. IEEE, 13–22.

Digital Library

[40]

Peter D. Mosses. 2004. Modular Structural Operational Semantics. J. Log. Algebr. Program. 60-61 (2004), 195–228.

[41]

Daejun Park, Andrei Stefănescu, and Grigore Roşu. 2015. KJS: A Complete Formal Semantics of JavaScript. In ACM SIGPLAN Notices, Vol. 50. ACM, 346–356.

Digital Library

[42]

Bernard James Pope. 2016. language-python: Parsing and pretty printing of Python code. http://hackage.haskell.org/ package/language-python-0.5.4 . (July 2016).

[43]

PUC-Rio. 2016. Lua: Test suites. Version 5.3.3. https://www.lua.org/tests/ . (2016).

[44]

Semantic Designs, Inc. 2005. Test Coverage tools. http://www.semanticdesigns.com/Products/TestCoverage/ . (2005).

[45]

Tim Sheard and Simon Peyton Jones. 2002. Template Meta-Programming for Haskell. In Proceedings of the 2002 ACM SIGPLAN Workshop on Haskell. ACM, 1–16.

Digital Library

[46]

Wouter Swierstra. 2008. Data Types à la Carte. Journal of Functional Programming 18, 04 (2008), 423–436.

Digital Library

[47]

ECMA TC39. 2014. Test262: ECMAScript Language Conformance Test Suite. Version 5.1. http://test262.ecmascript.org . (2014).

[48]

Danny van Bruggen. 2016. JavaParser: Process Java code programmatically. (2016). http://javaparser.org

[49]

Mark van de Brand, Jan Heering, Paul Klint, Ralf Lämmel, and Christian Verhoef. 2003. Language-Parametric Program Restructuring. (2003). http://www.cs.vu.nl/lppr/abstract/abstract.html

[50]

Jilles Van Gurp, Jan Bosch, and Mikael Svahnberg. 2001. On the Notion of Variability in Software Product Lines. In Software Architecture, 2001. Proceedings. Working IEEE/IFIP Conference on. IEEE, 45–54.

Digital Library

[51]

Markus Voelter and Vaclav Pech. 2012. Language Modularity with the MPS Language Workbench. In 2012 34th International Conference on Software Engineering (ICSE). IEEE, 1449–1450.

Digital Library

[52]

Philip Wadler. 1987. Views: A Way for Pattern Matching to Cohabit with Data Abstraction. In Proceedings of the 14th ACM SIGACT-SIGPLAN symposium on Principles of programming languages. ACM, 307–313.

Digital Library

[53]

Philip Wadler. 1998. The Expression Problem. Java-genericity mailing list (1998).

[54]

Stefan Wellek. 2010. Testing Statistical Hypotheses of Equivalence and Noninferiority. CRC Press.

[55]

Alexey Rodriguez Yakushev, Stefan Holdermans, Andres Löh, and Johan Jeuring. 2009. Generic Programming with Fixed Points for Mutually Recursive Datatypes. In Proceeding of the 14th ACM SIGPLAN International Conference on Functional Programming, ICFP 2009, Edinburgh, Scotland, UK, August 31 - September 2, 2009. 233–244.

Digital Library

[56]

Haoyuan Zhang, Zewei Chu, Bruno C. d. S. Oliveira, and Tijs van der Storm. 2015. Scrap Your Boilerplate with Object Algebras. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2015, part of SPLASH 2015, Pittsburgh, PA, USA, October 25-30, 2015. 127–146.

Digital Library

[57]

Alan Zimmerman. 2016. language-javascript: Parser for JavaScript. http://hackage.haskell.org/package/ language-javascript-0.6.0.9 . (November 2016).

Cited By

Matute GNi WBarik TCheung AChasins S(2024)Syntactic Code Search with Sequence-to-Tree Matching: Supporting Syntactic Search with Incomplete Code FragmentsProceedings of the ACM on Programming Languages10.1145/36564608:PLDI(2051-2072)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656460
Ketkar ARamos DClapp LBarik RRamanathan M(2024)A Lightweight Polyglot Code Transformation LanguageProceedings of the ACM on Programming Languages10.1145/36564298:PLDI(1288-1312)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656429
Koppel JKearl JSolar-Lezama A(2022)Automatically deriving control-flow graph generators from operational semanticsProceedings of the ACM on Programming Languages10.1145/35476486:ICFP(742-771)Online publication date: 31-Aug-2022
https://dl.acm.org/doi/10.1145/3547648
Show More Cited By

Index Terms

One tool, many languages: language-parametric transformation with incremental parametric syntax
1. Software and its engineering
  1. Software notations and tools
    1. Compilers
      1. Translator writing systems and compiler generators
    2. Formal language definitions
      1. Syntax

Recommendations

Incremental parametric syntax for multi-language transformation
SPLASH Companion 2017: Proceedings Companion of the 2017 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity

We present a new approach for building source-to-source transformations that can run on multiple programming languages, based on a new way of representing programs called incremental parametric syntax. We implement this approach in our Cubix system, and ...
Incremental concrete syntax for embedded languages
SAC '11: Proceedings of the 2011 ACM Symposium on Applied Computing

Embedded domain-specific languages (EDSLs) are known to improve the productivity of developers. However, for many domains no DSL implementation is available. Two important reasons are: First, the effort to implement embedded DSLs that provide the domain'...
Incremental concrete syntax for embedded languages with support for separate compilation

Embedded domain-specific languages (EDSLs) are known to improve the productivity of developers. However, for many domains no DSL implementation is available and two important reasons for this are: First, the effort to implement EDSLs that provide the ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages

Proceedings of the ACM on Programming Languages Volume 2, Issue OOPSLA

November 2018

1656 pages

EISSN:2475-1421

DOI:10.1145/3288538

Issue’s Table of Contents

Copyright © 2018 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2018

Published in PACMPL Volume 2, Issue OOPSLA

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
678
Total Downloads

Downloads (Last 12 months)82
Downloads (Last 6 weeks)17

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Matute GNi WBarik TCheung AChasins S(2024)Syntactic Code Search with Sequence-to-Tree Matching: Supporting Syntactic Search with Incomplete Code FragmentsProceedings of the ACM on Programming Languages10.1145/36564608:PLDI(2051-2072)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656460
Ketkar ARamos DClapp LBarik RRamanathan M(2024)A Lightweight Polyglot Code Transformation LanguageProceedings of the ACM on Programming Languages10.1145/36564298:PLDI(1288-1312)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656429
Koppel JKearl JSolar-Lezama A(2022)Automatically deriving control-flow graph generators from operational semanticsProceedings of the ACM on Programming Languages10.1145/35476486:ICFP(742-771)Online publication date: 31-Aug-2022
https://dl.acm.org/doi/10.1145/3547648
Koppel JJackson DKell SVerna D(2020)Demystifying dependenceProceedings of the 2020 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3426428.3426916(48-64)Online publication date: 18-Nov-2020
https://dl.acm.org/doi/10.1145/3426428.3426916
Premtoon VKoppel JSolar-Lezama ADonaldson ATorlak E(2020)Semantic code search via equational reasoningProceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3385412.3386001(1066-1082)Online publication date: 11-Jun-2020
https://dl.acm.org/doi/10.1145/3385412.3386001
Cambronero JDang TVasilakis NShen JWu JRinard MMasuhara HPetricek T(2019)Active learning for software engineeringProceedings of the 2019 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3359591.3359732(62-78)Online publication date: 23-Oct-2019
https://dl.acm.org/doi/10.1145/3359591.3359732
van Tonder RLe Goues CMcKinley KFisher K(2019)Lightweight multi-language syntax transformation with parser parser combinatorsProceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3314221.3314589(363-378)Online publication date: 8-Jun-2019
https://dl.acm.org/doi/10.1145/3314221.3314589
van Tonder RGoues CShihab EWagner S(2019)Towards s/engineer/botProceedings of the 1st International Workshop on Bots in Software Engineering10.1109/BotSE.2019.00019(43-47)Online publication date: 27-May-2019
https://dl.acm.org/doi/10.1109/BotSE.2019.00019

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents