PAVT: a tool to visualize and teach parsing algorithms

Abstract

A parsing algorithm visualizer is a tool that visualizes the construction of a parser for a given context-free grammar and then illustrates the use of that parser to parse a given string. Parsing algorithm visualizers are used to teach the course on compiler construction which in invariably included in all undergraduate computer science curricula. This paper presents a new parsing algorithm visualizer that can visualize six parsing algorithms, viz. predictive parsing, simple LR parsing, canonical LR parsing, look-ahead LR parsing, Earley parsing and CYK parsing. The tool logically explains the process of parsing showing the calculations involved in each step. The output of the tool has been structured to maximize the learning outcomes and contains important constructs like FIRST and FOLLOW sets, item sets, parsing table, parse tree and leftmost or rightmost derivation depending on the algorithm being visualized. The tool has been used to teach the course on compiler construction at both undergraduate and graduate levels. An overall positive feedback was received from the students with 89% of them saying that the tool helped them in understanding the parsing algorithms. The tool is capable of visualizing multiple parsing algorithms and 88% students used it to compare the algorithms.

APPENDIX

1.1 Predictive parsing

Given below is a sample output of the module to visualize the predictive parsing algorithm. We provide the grammar given in Section 2 and the string “(i + i)*i” as input. The grammar is left-recursive but does not have any scope for left-factoring. The module first displays the LL(1) grammar obtained after eliminating left-recursion. The module then displays FIRST sets for the 5 terminals and the 5 nonterminals, and FOLLOW sets for the 5 nonterminals. The module then displays the parsing table. The parsing table has 5 rows, one for each nonterminal, and 6 columns, one for each terminal and the ‘$’ character. A cell in the parsing table either contains a production rule or is empty. Then the module illustrates the table-driven parsing of the string. In each step, the part of the string that has been already matched, the content of the parsing stack, the part of the string yet to be matched and the production rule used in that step are displayed. The process ends with the acceptance of the string when the entire string has been matched and the parsing stack contains only the ‘$’ character. Since the parsing process has resulted in the acceptance of the string, the module displays the parse tree. Since predictive parsing follows leftmost derivation, the module at last displays the leftmost derivation of the string from the grammar start symbol.

1.2 Simple LR parsing

Given below is the output of the module to visualize the simple LR parsing algorithm for the grammar given in Section 2 and the string “(i + i)*i”. The module first displays the augmented grammar with Z being the new start symbol. The module then displays FIRST sets for the 5 terminals and the 4 nonterminals, and FOLLOW sets for the 4 nonterminals. The module then displays the item sets. The canonical collection contains 12 item sets, viz. I0 to I11. The module then displays the parsing table. The parsing table has 12 rows corresponding to the states constructed from the 12 item sets. The ACTION part of the parsing table has 6 columns, one for each terminal and the ‘$’ character, while the GOTO part of the parsing table has 3 columns, one for each nonterminal except Z. A cell in the ACTION part either specifies the action to be performed or is empty. A cell containing ‘sj’ specifies that the state j has to be pushed onto the parsing stack. A cell containing ‘rj’ specifies that the jth production rule has to be used to reduce. A cell in the GOTO part either gives the next state or is empty. The module then illustrates the table-driven parsing of the string. In each step, the content of the parsing stack, the part of the string yet to be matched, the action performed in that step and the production rule used, if the action was to reduce, are displayed. Since the parsing process results in the acceptance of the string, the module displays the parse tree. Note that the parse trees displayed by the modules visualizing predictive parsing and simple LR parsing are different. This is because of the different preprocessing techniques required by the two algorithms. Since simple LR parsing follows rightmost derivation in reverse, the module at last displays the rightmost derivation in reverse order.

1.3 Canonical LR parsing

Given below is the output of the module to visualize the canonical LR parsing algorithm for the grammar given in Section 2 and the string “(i + i)*i”. The module first displays the augmented grammar, with Z being the new start symbol, and then FIRST sets for the 5 terminals and the 4 nonterminals. The module then displays the item sets. The canonical collection contains 22 item sets, viz. I0 to I21. The module then displays the parsing table. The parsing table is like the one used for simple LR parsing except for the fact that it contains 22 rows. The number of rows in the parsing table can be more even by an order of magnitude in the case of canonical LR parsing than in the case of simple LR parsing for a satisfactorily large grammar. The module then illustrates the table-driven parsing of the string. The number of steps in the parsing process is exactly same as that in the case of simple LR parsing. The module displays the parse tree and the rightmost derivation in reverse order, which are again same as those in the case of simple LR parsing.

1.4 Look-ahead LR parsing

The output of the module to visualize the look-ahead LR parsing algorithm is similar to that of the module to visualize canonical LR parsing for the same grammar and the same string. However, due to the merging of item sets there are fewer rows in the parsing table. In fact, the number of rows in the parsing table is same for simple LR parsing and look-ahead LR parsing for a given grammar.

1.5 Earley parsing

Given below is the output of the module to visualize the Earley parsing algorithm for the grammar given in Section 2 and the string “(i + i)*i”. The module displays the augmented grammar, with Z being the new start symbol. The module then displays the 8 Earley item sets, viz. S[0] to S[7]. Since S[7] contains the item [Z- > E.,0], the string is accepted.

1.6 CYK parsing

Given below is the output of the module to visualize the CYK parsing algorithm for the same grammar and the same string. The module displays the grammar after it has been converted to CNF. As many as 8 new nonterminals has been introduced in the grammar, viz. R, S, U, V, W, X, Y and Z, with Z being the new start symbol. The module then displays the binary matrices. Since there are 7 terminals in the string, 7 binary matrices are printed. A value of 1 in the jth row and kth column of the ith binary matrix means that a string of i terminals can be derived from the jth nonterminal starting at the kth terminal. The string is accepted because the cell corresponding to Z and ‘(‘in the seventh binary matrix has a value of 1. The module also displays the same information using a lower triangular matrix. A nonterminal in the ith row from the bottom and the kth column in this lower triangular matrix can derive a string of i terminals starting with the kth. The presence of Z, the grammar start symbol, in the seventh row from the bottom and first column denotes the acceptance of the string.