research-article

Pure and declarative syntax definition: paradise lost and regained

Authors:

Lennart C.L. Kats,

Guido WachsmuthAuthors Info & Claims

OOPSLA '10: Proceedings of the ACM international conference on Object oriented programming systems languages and applications

Pages 918 - 932

https://doi.org/10.1145/1869459.1869535

Published: 17 October 2010 Publication History

Abstract

Syntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation algorithm. They hamper evolution, maintainability, and compositionality of syntax definitions. The pureness and declarativity of syntax definitions is lost. We analyze how these problems arise for different aspects of syntax definitions, discuss their consequences for language engineers, and show how the pure and declarative nature of syntax definitions can be regained.

References

[1]

}}A. V. Aho, S. C. Johnson, and J. D. Ullman. Deterministic parsing of ambiguous grammars. Commun. ACM, 18(8):441--452, 1975.

Digital Library

[2]

}}A. V. Aho and T. G. Peterson. A minimum distance error-correcting parser for context-free languages. SIAM J. Comput., 1(4):305--312, 1972.

[3]

}}A. Birman and J. D. Ullman. Parsing algorithms with backtrack. In Conference Record of 1970 Eleventh Annual Symposium on Switching and Automata Theory, 28-30 October 1970, Santa Monica, California, USA, pages 153--174. IEEE, 1970.

Digital Library

[4]

}}A. Borghi, V. David, and A. Demaille. C-Transformers: a framework to write C program transformations. ACM Crossroads, 12(3):3, 2005.

Digital Library

[5]

}}E. Bouwers, M. Bravenboer, and E. Visser. Grammar engineering support for precedence rule recovery and compatibility checking. In A. Sloane and A. Johnstone, editors, Seventh Workshop on Language Descriptions, Tools, and Applications (LDTA 2007), volume 203 of Electronic Notes in Theoretical Computer Science, pages 85--101, Braga, Portugal, March 2008. Elsevier.

Digital Library

[6]

}}M. G. J. van den Brand, H. de Jong, P. Klint, and P. Olivier. Efficient annotated terms. Software, Practice & Experience, 30(3):259--291, 2000.

Digital Library

[7]

}}M. G. J. van den Brand, J. Scheerder, J. Vinju, and E. Visser. Disambiguation filters for scannerless generalized LR parsers. In N. Horspool, editor, Compiler Construction (CC 2002), volume 2304 of Lecture Notes in Computer Science, pages 143--158, Grenoble, France, April 2002. Springer-Verlag.

Digital Library

[8]

}}M. G. J. van den Brand and E. Visser. Generation of formatters for context-free languages. TOSEM, 5(1):1--41, January 1996.

Digital Library

[9]

}}M. Bravenboer, E. Dolstra, and E. Visser. Preventing injection attacks with syntax embeddings. In C. Consel and J. L. Lawall, editors, Generative Programming and Component Engineering, 6th International Conference, GPCE 2007, pages 3--12, Salzburg, Austria, 2007. ACM.

Digital Library

[10]

}}M.Bravenboer,EricTanter,andE.Visser.Declarative,formal, and extensible syntax definition for AspectJ. In P. L. Tarr and W. R. Cook, editors, Proceedings of the 21th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2006, October 22-26, 2006, Portland, Oregon, USA, pages 209--228. ACM, 2006.

Digital Library

[11]

}}D. G. Cantor. On the ambiguity problem of backus systems. Journal of the ACM, 9(4):477--479, 1962.

Digital Library

[12]

}}P. Charles. A practical method for constructing efficient LALR(K) parsers with automatic error recovery. PhD thesis, New York University, 1991.

Digital Library

[13]

}}N. Chomsky. Syntactic Structures. Mouton de Gruyter, 1957.

[14]

}}J. Cocke. Programming languages and their compilers: Preliminary notes. Courant Institute of Mathematical Sciences, New York University, 1969.

Digital Library

[15]

}}J. R. Cordy, C. D. Halpern-Hamu, and E. Promislow. TXL: a rapid prototyping system for programming language dialects. In Conf. on Comp. Languages, pages 280--285. IEEE, 1988.

[16]

}}M. de Jonge. A pretty-printer for every occasion. In The International Symposium on Constructing Software Engineering Tools (CoSET2000), pages 68--77. University of Wollongong, Australia, 2000.

[17]

}}M. de Jonge, E. Nilsson-Nyman, L. C. L. Kats, and E. Visser. Natural and flexible error recovery for generated parsers. In M. van den Brand, D. Gasevic, and J. Gray, editors, Second International Conference, SLE 2009, Denver, CO, USA, October 5-6, 2009, Revised Selected Papers, Lecture Notes in Computer Science. Springer, 2010.

Digital Library

[18]

}}F. L. DeRemer. Practical translation for LR(k) languages. PhD thesis, MIT, Cambridge, MA, USA, September 1969.

[19]

}}J. Earley. An efficient context-free parsing algorithm. Commun. ACM, 13(2):94--102, 1970.

Digital Library

[20]

}}J.-M. Favre. Languages evolve too! Changing the software time scale. In 8th International Workshop on Principles of Software Evolution (IWPSE 2005), 5-7 September 2005, Lisbon, Portugal, pages 33--44. IEEE Computer Society, 2005.

Digital Library

[21]

}}R. Filman and D. Friedman. Aspect-oriented programming is quantification and obliviousness. In Workshop on Advanced separation of Concerns, 2000.

Digital Library

[22]

}}C. Fischer and R. LeBlanc. Crafting a compiler. Benjamin/Cummings Menlo Park, California, USA, 1988.

Digital Library

[23]

}}B. Ford. Packrat parsing: simple, powerful, lazy, linear time, functional pearl. In Proceedings of the seventh ACM SIGPLAN international conference on Functional Programming (ICFP 2002), pages 36--47, 2002.

Digital Library

[24]

}}B. Ford. Parsing expression grammars: a recognition-based syntactic foundation.In N. D. Jones and X. Leroy, editors, Proceedings of the 31st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2004, Venice, Italy, January 14-16, 2004, pages 111--122. ACM, 2004.

Digital Library

[25]

}}S. Ginsburg and J. Ullian. Ambiguity in context free languages. J. ACM, 13(1):62--89, 1966.

Digital Library

[26]

}}S. Gorn. Detection of generative ambiguities in context-free mechanical languages. J. ACM, 10(2):196--208, 1963.

Digital Library

[27]

}}S. L. Graham, C. B. Haley, and W. N. Joy. Practical LR error recovery. In Proceedings of the 1979 SIGPLAN Symposium on Compiler Construction, Denver, Colorado, USA, August 6-10, 1979, pages 168--175. ACM, 1979.

Digital Library

[28]

}}J. Heering, P. R. H. Hendriks, P. Klint, and J. Rekers. The syntax definition formalism SDF - reference manual. SIGPLAN Notices, 24(11):43--75, 1989.

Digital Library

[29]

}}S. C. Johnson. YACC - yet another compiler-compiler. Technical Report CS-32, AT & T Bell Laboratories, Murray Hill, N.J., 1975.

[30]

}}A. Johnstone, E. Scott, and G. Economopoulos. Evaluating GLR parsing algorithms. Science of Computer Programming, 61(3):228--244, 2006.

Digital Library

[31]

}}T. Kasami. An efficient recognition and syntax analysis algorithm for context free languages. Science Report AF CRL-65-758, Air Force Cambridge Research Laboratory, Bedford, Mass., USA, 1965.

[32]

}}L. C. L. Kats, M. de Jonge, E. Nilsson-Nyman, and E. Visser. Providing rapid feedback in generated modular language environments. Adding error recovery to scannerless generalized-LR parsing. In S. Arora and G. T. Leavens, editors, Proceedings of the 24th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2009, October 25--29, 2009, Orlando, Florida, USA., pages 445--464, 2009.

Digital Library

[33]

}}L. C. L. Kats and E. Visser. The Spoofax language workbench. Rules for declarative specification of languages and IDEs. In M. Rinard, editor, Proceedings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2010, October 17-21, 2010, Reno, NV, USA, 2010.

Digital Library

[34]

}}P. Klint, R. Lammel, and C. Verhoef. Toward an engineering discipline for grammarware. ACM Transactions on Software Engineering Methodology, 14(3):331--380, 2005.

Digital Library

[35]

}}P. Klint and E. Visser. Using filters for the disambiguation of context-free grammars. In Proc. ASMICS Workshop on Parsing Theory, pages 1--20, Milano, Italy, October 1994. Tech. Rep. 126-1994, Dipartimento di Scienze dell'Informazione, Universita di Milano.

[36]

}}D. E. Knuth. On the translation of languages from left to right. Information and control, 8(6):607--639, 1965.

[37]

}}P. M. Lewis II and R. E. Stearns. Syntax-directed transduction. Journal of the ACM, 15(3):465--488, 1968.

Digital Library

[38]

}}B. A. Malloy, J. F. Power, and J. T. Waldron. Applying software engineering techniques to parser design: the development of a C# parser. In SAICSIT '02: Proceedings of the 2002 annual research conference of the South African institute of computer scientists and information technologists on Enablement through technology, pages 75--82, Port Elizabeth, Republic of South Africa, 2002. South African Institute for Computer Scientists and Information Technologists.

Digital Library

[39]

}}P. M. Maurer. Generating test data with enhanced context-free grammars. IEEE Software, 7(4):50--55, 1990.

Digital Library

[40]

}}Panini Astadhyay1. Otto Bohtlingk, editor, Konig, 1839--1840.

[41]

}}Panini Astadhyay1. Benares, 1896. Translated by Sr1sa Chandra Vasu.

[42]

}}T. J. Parr. ANTLR Parser Generator. http://www.antlr.org/.

[43]

}}T. J. Parr and R. W. Quong. LL and LR translators need k>1 lookahead. SIGPLAN Not., 31(2):27--34, 1996.

Digital Library

[44]

}}J. Rekers. Parser Generation for Interactive Environments. PhD thesis, University of Amsterdam, Amsterdam, The Netherlands, January 1992.

[45]

}}D. Salomon and G. Cormack. The disambiguation and scannerless parsing of complete character-level grammars for programming languages. Technical Report 95/06, Department of Computer Science, University of Manitoba, Winnipeg, Canada, 1995.

[46]

}}D. J. Salomon and G. V. Cormack. Scannerless NSLR(1) parsing of programming languages. SIGPLAN Not., 24(7):170--178, 1989.

Digital Library

[47]

}}E. Scott and A. Johnstone. GLL parsing. In Workshop on Language Descriptions, Tools and Applications (LDTA'09), 2009.

[48]

}}The Spoofax project. http://www.spoofax.org/.

[49]

}}M. Tomita. Efficient Parsing for Natural Language: A Fast Algorithm for Practical Systems, volume 14. Kluwer Academic Publishers, 1988.

Digital Library

[50]

}}E. Visser. A case study in optimizing parsing schemata by disambiguation filters. In IWPT, pages 210--224, 1997.

[51]

}}E. Visser. Scannerless generalized-LR parsing. Technical Report P9707, Programming Research Group, University of Amsterdam, July 1997.

[52]

}}E. Visser. Syntax Definition for Language Prototyping. PhD thesis, University of Amsterdam, September 1997.

[53]

}}E. Visser. Meta-programming with concrete object syntax. In D. S. Batory, C. Consel, and W. Taha, editors, Generative Programming and Component Engineering, ACM SIGPLAN/SIGSOFT Conference, GPCE 2002, Pittsburgh, PA, USA, October 6-8, 2002, Proceedings, volume 2487 of Lecture Notes in Computer Science, pages 299--315. Springer, 2002.

Digital Library

[54]

}}D. Younger. Recognition and parsing of context-free languages in time n3. Information and control, 10(2):189--208, 1967.

Cited By

Li LTaura K(2023)Associative Operator Precedence Parsing: A Method To Increase Data Parsing ParallelismProceedings of the International Conference on High Performance Computing in Asia-Pacific Region10.1145/3578178.3578233(75-87)Online publication date: 27-Feb-2023
https://dl.acm.org/doi/10.1145/3578178.3578233
Boukham HWachsmuth GDwars MChiadmi DFischer BBurgueño LCazzola W(2022)A Multi-target, Multi-paradigm DSL Compiler for Algorithmic Graph ProcessingProceedings of the 15th ACM SIGPLAN International Conference on Software Language Engineering10.1145/3567512.3567513(2-15)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3567512.3567513
de Souza Amorim LVisser E(2020)Multi-purpose Syntax Definition with SDF3Software Engineering and Formal Methods10.1007/978-3-030-58768-0_1(1-23)Online publication date: 8-Sep-2020
https://doi.org/10.1007/978-3-030-58768-0_1
Show More Cited By

Index Terms

Pure and declarative syntax definition: paradise lost and regained
1. Software and its engineering
  1. Software notations and tools
    1. Compilers
      1. Parsers
    2. Formal language definitions
      1. Syntax

Recommendations

Pure and declarative syntax definition: paradise lost and regained
OOPSLA '10

Syntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation ...
Declarative, formal, and extensible syntax definition for aspectJ
Proceedings of the 2006 OOPSLA Conference

Aspect-Oriented Programming (AOP) is attracting attention from both research and industry, as illustrated by the ever-growing popularity of AspectJ, the de facto standard AOP extension of Java. From a compiler construction perspective AspectJ is ...
Integrated language definition testing: enabling test-driven language development
OOPSLA '11: Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications

The reliability of compilers, interpreters, and development environments for programming languages is essential for effective software development and maintenance. They are often tested only as an afterthought. Languages with a smaller scope, such as ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

OOPSLA '10: Proceedings of the ACM international conference on Object oriented programming systems languages and applications

October 2010

984 pages

ISBN:9781450302036

DOI:10.1145/1869459

General Chairs:
William R. Cook
University of Texas at Austin
,
Siobhán Clarke
Trinity College Dublin
,
Program Chairs:
Martin Rinard
MIT
,
Kevin J. Sullivan
University of Virginia
,
Daniel H. Steinberg
Dim Sum Thinking Inc.

ACM SIGPLAN Notices Volume 45, Issue 10
OOPSLA '10
October 2010
957 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/1932682
Issue’s Table of Contents

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SPLASH '10

Sponsor:

SIGPLAN

SPLASH '10: Systems Programming Languages and Applications: Software for Humanity

October 17 - 21, 2010

Nevada, Reno/Tahoe, USA

Acceptance Rates

Overall Acceptance Rate 268 of 1,244 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

64
Total Citations
View Citations
689
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)1

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li LTaura K(2023)Associative Operator Precedence Parsing: A Method To Increase Data Parsing ParallelismProceedings of the International Conference on High Performance Computing in Asia-Pacific Region10.1145/3578178.3578233(75-87)Online publication date: 27-Feb-2023
https://dl.acm.org/doi/10.1145/3578178.3578233
Boukham HWachsmuth GDwars MChiadmi DFischer BBurgueño LCazzola W(2022)A Multi-target, Multi-paradigm DSL Compiler for Algorithmic Graph ProcessingProceedings of the 15th ACM SIGPLAN International Conference on Software Language Engineering10.1145/3567512.3567513(2-15)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3567512.3567513
de Souza Amorim LVisser E(2020)Multi-purpose Syntax Definition with SDF3Software Engineering and Formal Methods10.1007/978-3-030-58768-0_1(1-23)Online publication date: 8-Sep-2020
https://doi.org/10.1007/978-3-030-58768-0_1
Amorim LSteindorfer MErdweg SVisser EPearce DMayerhofer TSteimann F(2018)Declarative specification of indentation rules: a tooling perspective on parsing and pretty-printing layout-sensitive languagesProceedings of the 11th ACM SIGPLAN International Conference on Software Language Engineering10.1145/3276604.3276607(3-15)Online publication date: 24-Oct-2018
https://dl.acm.org/doi/10.1145/3276604.3276607
Kuramitsu KCombemale BMernik MRumpe B(2017)A symbol-based extension of parsing expression grammars and context-sensitive packrat parsingProceedings of the 10th ACM SIGPLAN International Conference on Software Language Engineering10.1145/3136014.3136025(26-37)Online publication date: 23-Oct-2017
https://dl.acm.org/doi/10.1145/3136014.3136025
Amorim LSteindorfer MVisser ECombemale BMernik MRumpe B(2017)Deep priority conflicts in the wild: a pilot studyProceedings of the 10th ACM SIGPLAN International Conference on Software Language Engineering10.1145/3136014.3136020(55-66)Online publication date: 23-Oct-2017
https://dl.acm.org/doi/10.1145/3136014.3136020
Kuramitsu K(2016)Fast, Flexible, and Declarative Construction of Abstract Syntax Trees with PEGsJournal of Information Processing10.2197/ipsjjip.24.12324:1(123-131)Online publication date: 2016
https://doi.org/10.2197/ipsjjip.24.123
Kuramitsu KVisser EMurphy-Hill ELopes C(2016)Nez: practical open grammar languageProceedings of the 2016 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/2986012.2986019(29-42)Online publication date: 20-Oct-2016
https://dl.acm.org/doi/10.1145/2986012.2986019
Afroozeh AIzmaylova AZaks AHermenegildo M(2016)Iguana: a practical data-dependent parsing frameworkProceedings of the 25th International Conference on Compiler Construction10.1145/2892208.2892234(267-268)Online publication date: 17-Mar-2016
https://dl.acm.org/doi/10.1145/2892208.2892234
Afroozeh AIzmaylova AErwig MRompf T(2016)Operator precedence for data-dependent grammarsProceedings of the 2016 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation10.1145/2847538.2847540(13-24)Online publication date: 11-Jan-2016
https://dl.acm.org/doi/10.1145/2847538.2847540
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents