research-article

LL(*): the foundation of the ANTLR parser generator

Authors:

Kathleen FisherAuthors Info & Claims

PLDI '11: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation

Pages 425 - 436

https://doi.org/10.1145/1993498.1993548

Published: 04 June 2011 Publication History

Abstract

Despite the power of Parser Expression Grammars (PEGs) and GLR, parsing is not a solved problem. Adding nondeterminism (parser speculation) to traditional LL and LR parsers can lead to unexpected parse-time behavior and introduces practical issues with error handling, single-step debugging, and side-effecting embedded grammar actions. This paper introduces the LL(*) parsing strategy and an associated grammar analysis algorithm that constructs LL(*) parsing decisions from ANTLR grammars. At parse-time, decisions gracefully throttle up from conventional fixed k>=1 lookahead to arbitrary lookahead and, finally, fail over to backtracking depending on the complexity of the parsing decision and the input symbols. LL(*) parsing strength reaches into the context-sensitive languages, in some cases beyond what GLR and PEGs can express. By statically removing as much speculation as possible, LL(*) provides the expressivity of PEGs while retaining LL's good error handling and unrestricted grammar actions. Widespread use of ANTLR (over 70,000 downloads/year) shows that it is effective for a wide variety of applications.

References

[1]

Bermudez, M. E., and Schimpf, K. M. Practical arbitrary lookahead LR parsing. Journal of Computer and System Sciences 41, 2 (1990), 230--250.

[2]

Charles, P. A Practical Method for Constructing Efficient LALR(K) Parsers with Automatic Error Recovery. PhD thesis, New York University, New York, NY, USA, 1991.

Digital Library

[3]

Cohen, R., and Culik, K. LR-Regular grammars an extension of LR(k) grammars. In SWAT '71: Proceedings of the 12th Annual Symposium on Switching and Automata Theory (swat 1971) (Washington, DC, USA, 1971), IEEE Computer Society, pp. 153--165.

Digital Library

[4]

Earley, J. An efficient context-free parsing algorithm. Communications of the ACM 13, 2 (1970), 94--102.

Digital Library

[5]

Ford, B. Packrat Parsing: Simple, powerful, lazy, linear time. In Proceedings of annual ACM SIGPLAN International Conference on Functional Programming (2002), ACM Press, pp. 36--47.

Digital Library

[6]

Ford, B. Parsing Expression Grammars: A recognition-based syntactic foundation. In POPL '04: Proceedings of the 37th annual ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages (2004), ACM Press, pp. 111--122.

Digital Library

[7]

Grimm, R. Better extensibility through modular syntax. In PLDI'06: Proceedings of annual ACM SIGPLAN Conference on Programming Language Design and Implementation (2006), ACM Press, pp. 38--51.

Digital Library

[8]

Hanson, D. R. Compact recursive-descent parsing of expressions. Software Practice and Experience 15 (December 1985), 1205--1212.

Digital Library

[9]

Jarzabek, S., and Krawczyk, T. LL-Regular grammars. Information Processing Letters 4, 2 (1975), 31--37.

[10]

Jim, T., Mandelbaum, Y., and Walker, D. Semantics and algorithms for data-dependent grammars. In POPL '10: Proceedings of the 37th annual ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages (New York, NY, USA, 2010), ACM, pp. 417--430.

Digital Library

[11]

McPeak, S., and Necula, G. C. Elkhound: A fast, practical GLR parser generator. In Compiler Construction (2004), pp. 73--88.

[12]

Milton, D. R., and Fischer, C. N. LL(k) parsing for attributed grammars. In International Conference on Automata, Languages, and Programming (1979), pp. 422--430.

Digital Library

[13]

Nederhof, M.-J. Practical experiments with regular approximation of context-free languages. Computational Linguistics 26, 1 (2000), 17--44.

Digital Library

[14]

Nijholt, A. On the parsing of LL-Regular grammars. In Mathematical Foundations of Computer Science 1976 (Heidelberg, 1976), A. Mazurkiewicz, Ed., vol. 45 of Lecture Notes in Computer Science, Springer Verlag, pp. 446--452.

[15]

Nijholt, A. On the parsing of LL-Regular grammars. In Mathematical Foundations of Computer Science 1976 (Heidelberg, 1976), A. Mazurkiewicz, Ed., vol. 45 of Lecture Notes in Computer Science, Springer Verlag, pp. 446--452.

[16]

Parr, T. J. Obtaining practical variants of LL(k) and LR(k) for k > 1 by splitting the atomic k-tuple. PhD thesis, Purdue University, West Lafayette, IN, USA, 1993.

Digital Library

[17]

Parr, T. J., and Quong, R. W. Adding Semantic and Syntactic Predicates to LL(k)|pred-LL(k). In Proceedings of the International Conference on Compiler Construction; Edinburgh, Scotland (April 1994).

Digital Library

[18]

Poplawski, D. A. On LL-Regular grammars. Journal of Computer and System Sciences 18, 3 (1979), 218--227.

[19]

Tomita, M. Efficient Parsing for Natural Language. Kluwer Academic Publishers, 1986.

Digital Library

[20]

Woods, W. A. Transition network grammars for natural language analysis. Communications of the ACM 13, 10 (1970), 591--606.

Digital Library

Cited By

Blakey MPearman-Kanza SFrey J(2024)Zombie cheminformatics: extraction and conversion of Wiswesser Line Notation (WLN) from chemical documentsJournal of Cheminformatics10.1186/s13321-024-00831-216:1Online publication date: 15-Apr-2024
https://doi.org/10.1186/s13321-024-00831-2
Santos ACancelinha ABatista FDig DBryksin TGolubev YBezzubov A(2024)Jasay: Towards Voice Commands in Projectional EditorsProceedings of the 1st ACM/IEEE Workshop on Integrated Development Environments10.1145/3643796.3648449(30-34)Online publication date: 20-Apr-2024
https://dl.acm.org/doi/10.1145/3643796.3648449
Godboley SKrishna PJha R(2024)Atomic Condition Coverage Analysis for Structured Text Based Programmable Logic Controller (PLC)Proceedings of the 17th Innovations in Software Engineering Conference10.1145/3641399.3641427(1-5)Online publication date: 22-Feb-2024
https://dl.acm.org/doi/10.1145/3641399.3641427
Show More Cited By

Index Terms

LL(*): the foundation of the ANTLR parser generator
1. Software and its engineering
  1. Software notations and tools
    1. Compilers
      1. Parsers
    2. Formal language definitions
      1. Syntax
2. Theory of computation
  1. Semantics and reasoning
    1. Program reasoning
      1. Parsing

Recommendations

Adaptive LL(*) parsing: the power of dynamic analysis
OOPSLA '14

Despite the advances made by modern parsing strategies such as PEG, LL(*), GLR, and GLL, parsing is not a solved problem. Existing approaches suffer from a number of weaknesses, including difficulties supporting side-effecting embedded actions, slow and/...
LL(*): the foundation of the ANTLR parser generator
PLDI '11

Despite the power of Parser Expression Grammars (PEGs) and GLR, parsing is not a solved problem. Adding nondeterminism (parser speculation) to traditional LL and LR parsers can lead to unexpected parse-time behavior and introduces practical issues with ...
Adaptive LL(*) parsing: the power of dynamic analysis
OOPSLA '14: Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications

Despite the advances made by modern parsing strategies such as PEG, LL(*), GLR, and GLL, parsing is not a solved problem. Existing approaches suffer from a number of weaknesses, including difficulties supporting side-effecting embedded actions, slow and/...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PLDI '11: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation

June 2011

668 pages

ISBN:9781450306638

DOI:10.1145/1993498

General Chair:
Mary Hall
University of Utah
,
Program Chair:
David Padua
University of Illinois at Urbana-Champaign

ACM SIGPLAN Notices Volume 46, Issue 6
PLDI '11
June 2011
652 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/1993316
Issue’s Table of Contents

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 June 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

PLDI '11

Sponsor:

SIGPLAN

PLDI '11: ACM SIGPLAN Conference on Programming Language Design and Implementation

June 4 - 8, 2011

California, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 406 of 2,067 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

134
Total Citations
View Citations
2,390
Total Downloads

Downloads (Last 12 months)125
Downloads (Last 6 weeks)14

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Blakey MPearman-Kanza SFrey J(2024)Zombie cheminformatics: extraction and conversion of Wiswesser Line Notation (WLN) from chemical documentsJournal of Cheminformatics10.1186/s13321-024-00831-216:1Online publication date: 15-Apr-2024
https://doi.org/10.1186/s13321-024-00831-2
Santos ACancelinha ABatista FDig DBryksin TGolubev YBezzubov A(2024)Jasay: Towards Voice Commands in Projectional EditorsProceedings of the 1st ACM/IEEE Workshop on Integrated Development Environments10.1145/3643796.3648449(30-34)Online publication date: 20-Apr-2024
https://dl.acm.org/doi/10.1145/3643796.3648449
Godboley SKrishna PJha R(2024)Atomic Condition Coverage Analysis for Structured Text Based Programmable Logic Controller (PLC)Proceedings of the 17th Innovations in Software Engineering Conference10.1145/3641399.3641427(1-5)Online publication date: 22-Feb-2024
https://dl.acm.org/doi/10.1145/3641399.3641427
Choi KHwang SMoon HSasano IHong JPark J(2024)Ranked Syntax Completion With LR ParsingProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3635944(1242-1251)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1145/3605098.3635944
van der Merwe B(2023)Ordered Context-Free Grammars RevisitedElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.388.13388(140-153)Online publication date: 15-Sep-2023
https://doi.org/10.4204/EPTCS.388.13
Miyazaki TMinamide Y(2023)Derivatives of Context-free Grammars with LookaheadJournal of Information Processing10.2197/ipsjjip.31.42131(421-431)Online publication date: 2023
https://doi.org/10.2197/ipsjjip.31.421
Zhang QFang CMa YSun WChen Z(2023)A Survey of Learning-based Automated Program RepairACM Transactions on Software Engineering and Methodology10.1145/363197433:2(1-69)Online publication date: 23-Dec-2023
https://dl.acm.org/doi/10.1145/3631974
Scott EJohnstone AWalsh R(2023)Multiple Input Parsing and Lexical AnalysisACM Transactions on Programming Languages and Systems10.1145/359473445:3(1-44)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3594734
Li LTaura K(2023)Associative Operator Precedence Parsing: A Method To Increase Data Parsing ParallelismProceedings of the International Conference on High Performance Computing in Asia-Pacific Region10.1145/3578178.3578233(75-87)Online publication date: 27-Feb-2023
https://dl.acm.org/doi/10.1145/3578178.3578233
Palmkvist VCastegren EHaller PBroman D(2023)Statically Resolvable AmbiguityProceedings of the ACM on Programming Languages10.1145/35712517:POPL(1686-1712)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3571251
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents