article

Agile Parsing in TXL

Authors:

Thomas R. Dean,

James R. Cordy,

Andrew J. Malton,

Kevin A. SchneiderAuthors Info & Claims

Automated Software Engineering, Volume 10, Issue 4

Pages 311 - 336

https://doi.org/10.1023/A:1025801405075

Published: 01 October 2003 Publication History

Abstract

Syntactic analysis forms a foundation of many source analysis and reverse engineering tools. However, a single standard grammar is not always appropriate for all source analysis and manipulation tasks. Small custom modifications to the grammar can make the programs used to implement these tasks simpler, clearer and more efficient. This leads to a new paradigm for programming these tools: agile parsing. In agile parsing the effective grammar used by a particular tool is a combination of two parts: the standard base grammar for the input language, and a set of explicit grammar overrides that modify the parse to support the task at hand. This paper introduces the basic techniques of agile parsing in TXL and discusses several industry proven techniques for exploiting agile parsing in software source analysis and transformation.

References

[1]

Badros, G. 2000. JavaML: A markup language for Java source code. Computer Networks, 33(1-6):159-177.

Digital Library

[2]

Baxter, I.D. and Pidgeon, C.W. 1997. Software change through design maintenance. Proc. 1997 International Conference on Software Maintenance, Bari, Italy, pp. 250-259.

Digital Library

[3]

Bell Canada. (2000). Datrix Abstract Semantic Graph: Reference Manual, Version 1.4, Bell Canada Inc., Montreal Canada, May 01.

[4]

van den Brand, M., Sellink, A., and Verhoef, C. 1998. Current parsing techniques in software renovation considered harmful. Proc. 6th International Workshop on Program Comprehension (IWPC 98), Ischia, Italy, pp. 108-117.

Digital Library

[5]

van den Brand, M., van Deursen, A., Heering, J., de Jong, H., de Jonge, M., Kuipers, T., Klint, P., Moonen, L., Olivier, P., Scheerder, J., Vinju, J., Visser, C., and Visser, J. 2001. The ASF+SDF Meta-Environment: A component-based language development environment. In R. Wilhelm, editors, Compiler Construction 2001 (CC 2001), Lecture Notes in Computer Science, Springer Verlag, vol. 1827, pp. 365-370.

[6]

van den Brand, M., Heering, J., Klint, P., and Olivier, P. 2002. Compiling rewrite systems: The ASF+SDF compiler. ACM Transactions on Programming Languages and Systems, 24(4): 334-368.

Digital Library

[7]

Cordy, J.R., Halpern, C.D., and Promislow, E. 1991. TXL: A rapid prototyping system for programming language dialects. Computer Languages, 16(1): 97-107.

Digital Library

[8]

Cordy, J.R., Carmichael, I.H., and Halliday, R. 2000. The TXL programming language--Version 10, Kingston: Queen's University at Kingston and Legasys Corporation.

[9]

Cordy, J., Schneider, K., Dean, T., and Malton, A. 2001. HSML: Design directed source code hot spots. Proc. 9th International Workshop on Program Comprehension (IWPC 01), Toronto, Canada, pp. 145-154.

Digital Library

[10]

Cordy, J., Dean, T., Malton, A., and Schneider, K. 2002. Source transformation in software engineering using the TXL transformation system. Special Issue on Source Code Analysis and Manipulation. Journal of Information and Software Technology, 44(13): 827-837.

[11]

Cordy, J. 2003. Generalized selective XML markup of source code using agile parsing. Proc. 11th International Workshop on Program Comprehension (IWPC 03), Portland, pp. 144-153.

Digital Library

[12]

Cox, A. and Clarke, C. 2000. A comparative evaluation of techniques for syntactic level source code analysis. Proc. 7th Asia-Pacific Software Engineering Conference (APSEC'00), Singapore, pp. 282-291.

Digital Library

[13]

Dean, T., Cordy, J., Schneider, K., and Malton, A. 2001. Experience using design recovery techniques to transform legacy systems. Proc. International Conference on Software Maintenance (ICSM 2001), Florence, Italy, pp. 622-631.

Digital Library

[14]

van Deursen, A. and Kuipers, T. 1999. Building documentation generators. Proc. International Conference on Software Maintenance (ICSM 99), Oxford, England, pp. 40-49.

Digital Library

[15]

Guo, X., Cordy, J., and Dean, T. 2003. Unique renaming of Java code using source transformation. Proc. 10th International Workshop on Source Code Analysis and Manipulation (SCAM 2003), Amsterdam, to appear.

[16]

Johnson, S.C. 1975. Yacc: Yet another compiler-compiler. Computing Science Technical Report No. 32. Murray Hill, N.J.: Bell Laboratories.

[17]

Lamb, D. and Schneider, K. 1992. Formalization of information hiding design methods. Proc. IBM Center for Advanced Studies Conference (CASCON '92), Toronto, Canada, pp. 201-214.

Digital Library

[18]

Lämmel, R. 2001. Grammar adaptation. Proc. International Symposium on Formal Methods Europe (FME'01), Berlin, pp. 550-570.

[19]

Lämmel, R. and Verhoef, C. 2001. Semi-automatic Grammar Recovery. Software Practice & Experience, 31(15): 1395-1438.

Digital Library

[20]

Lethbridge, T., Plödereder, E., Tichelaar, S., Riva, C., and Linos, P. 2001. The Dagstuhl Middle Model (DMM), Version 0.003, http://scgwiki.iam.unibe.ch:8080/Exchange/2.

[21]

Lesk, M.E. and Schmidt, E. 1975. Lex--A Lexical Analyzer Generator. Computing Science Technical Report No. 39. Murray Hill, NJ: Bell Laboratories.

[22]

Maletic, J., Collard, M., and Marcus, A. 2002. Source code files as structured documents. Proc. 10th International Workshop on Program Comprehension (IWPC 02), Paris, France, pp. 289-292.

Digital Library

[23]

Malton, A.J., Schneider, K.A., Cordy, J.R., Dean, T.R., Cousineau, D., and Reynolds, J. 2001. Processing software source text in automated design recovery and transformation. Proc. 9th International Workshop on Program Comprehension (IWPC 2001), Toronto, Canada, pp. 127-134.

Digital Library

[24]

Moonen, L. 2001. Generating robust parsers using island grammars. Proc. 8th Working Conference on Reverse Engineering (WCRE 01), Stuttgart, Germany, pp 13-22.

Digital Library

[25]

Moonen, L. 2002. Lightweight impact using island grammars. Proceedings 10th International Workshop on Program Comprehension (IWPC 02), Paris France, pp. 343-352.

Digital Library

[26]

Müller, H. and Klashinsky, K. 1988. Rigi--A System for Programming-in-the-Large. Proc. 10th International Conference on Software Engineering (ICSE 88), Singapore, pp. 80-86.

Digital Library

[27]

Neighbors, J. 1984. The Draco approach to constructing software from reusable components. IEEE Transactions on Software Engineering, 10(5): 564-574.

Digital Library

[28]

Reasoning Systems. 1992. Refine User's Manual. Palo Alto, California.

[29]

Sellink, A. and Verhoef, C. 1998. Native patterns. Proc. 5th Working Conference on Reverse Engineering, Honolulu, Hawaii, pp. 89-103.

Digital Library

[30]

Sellink, A. and Verhoef, C. 1999. An architecture for automated software maintenance. Proc. 7th International Workshop on Program Comprehension (IWPC 99), Pittsburgh, Pennsylvania, pp. 38-48.

Digital Library

[31]

TXL Project. 2003. FreeTXL 10.2e. http://www.txl.ca/ndownload.html.

[32]

Visser, E. 2001. Stratego: A language for program transformation based on rewriting strategies. In A. Middeldorp, editor, System Description of Stratego 0.5. Rewriting Techniques and Applications (RTA '01), Lecture Notes in Computer Science, Springer Verlag, pp. 357-361.

Cited By

Alalfi MAntony ECordy J(2018)An approach to clone detection in sequence diagrams and its application to security analysisSoftware and Systems Modeling (SoSyM)10.1007/s10270-016-0557-617:4(1287-1309)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s10270-016-0557-6
Bacchelli AMocci ACleve ALanza M(2017)Mining structured data in natural language artifacts with island parsingScience of Computer Programming10.1016/j.scico.2017.06.009150:C(31-55)Online publication date: 15-Dec-2017
https://dl.acm.org/doi/10.1016/j.scico.2017.06.009
Martin DCordy JAdams BAntoniol GDe Lucia ABird COliveto R(2015)Make it simpleProceedings of the 2015 IEEE 23rd International Conference on Program Comprehension10.5555/2820282.2820313(207-217)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2820282.2820313
Show More Cited By

Index Terms

Agile Parsing in TXL
1. Software and its engineering
  1. Software notations and tools
    1. Compilers
      1. Parsers
2. Theory of computation
  1. Logic
    1. Automated reasoning
  2. Semantics and reasoning
    1. Program reasoning
      1. Program analysis
    2. Program semantics

Recommendations

Transition network grammars for natural language analysis

The use of augmented transition network grammars for the analysis of natural language sentences is described. Structure-building actions associated with the arcs of the grammar network allow for the reordering, restructuring, and copying of constituents ...
Left recursion in Parsing Expression Grammars

Parsing Expression Grammars (PEGs) are a formalism that can describe all deterministic context-free languages through a set of rules that specify a top-down parser for some language. PEGs are easy to use, and there are efficient implementations of PEG ...
Parse views with Boolean grammars

We propose an enhancement to current parsing and transformation systems by leveraging the expressive power of Boolean grammars, a generalization of context-free grammars that adds conjunction and negation operators. In addition to naturally expressing a ...

Comments

Information & Contributors

Information

Published In

cover image Automated Software Engineering

Automated Software Engineering Volume 10, Issue 4

October 2003

91 pages

ISSN:0928-8910

Issue’s Table of Contents

Copyright © Copyright © 2003 Kluwer Academic Publishers.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 October 2003

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alalfi MAntony ECordy J(2018)An approach to clone detection in sequence diagrams and its application to security analysisSoftware and Systems Modeling (SoSyM)10.1007/s10270-016-0557-617:4(1287-1309)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s10270-016-0557-6
Bacchelli AMocci ACleve ALanza M(2017)Mining structured data in natural language artifacts with island parsingScience of Computer Programming10.1016/j.scico.2017.06.009150:C(31-55)Online publication date: 15-Dec-2017
https://dl.acm.org/doi/10.1016/j.scico.2017.06.009
Martin DCordy JAdams BAntoniol GDe Lucia ABird COliveto R(2015)Make it simpleProceedings of the 2015 IEEE 23rd International Conference on Program Comprehension10.5555/2820282.2820313(207-217)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2820282.2820313
Stevenson ACordy J(2015)Parse views with Boolean grammarsScience of Computer Programming10.1016/j.scico.2013.11.00797:P1(59-63)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.1016/j.scico.2013.11.007
Kurš JLungu MIyadurai RNierstrasz O(2015)Bounded seasComputer Languages, Systems and Structures10.1016/j.cl.2015.08.00244:PA(114-140)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.1016/j.cl.2015.08.002
Lämmel RZaytsev V(2011)Recovering grammar relationships for the Java Language SpecificationSoftware Quality Journal10.1007/s11219-010-9116-519:2(333-378)Online publication date: 1-Jun-2011
https://dl.acm.org/doi/10.1007/s11219-010-9116-5
Durán FRoldán MBach JBalland EVan Den Brand MCordy JEker SEngelen LDe Jonge MKalleberg KKats LMoreau PVisser E(2010)The third rewrite engines competitionProceedings of the 8th international conference on Rewriting logic and its applications10.5555/1927806.1927829(243-261)Online publication date: 20-Mar-2010
https://dl.acm.org/doi/10.5555/1927806.1927829
Kam BDean T(2010)Linguistic security testing for text communication protocolsProceedings of the 5th international academic and industrial conference on Testing - practice and research techniques10.5555/1885930.1885942(104-117)Online publication date: 3-Sep-2010
https://dl.acm.org/doi/10.5555/1885930.1885942
Cordy J(2009)Excerpts from the TXL cookbookProceedings of the 3rd international summer school conference on Generative and transformational techniques in software engineering III10.5555/1949925.1949928(27-91)Online publication date: 6-Jul-2009
https://dl.acm.org/doi/10.5555/1949925.1949928
Ceccato MDean TTonella P(2009)Recovering structured data types from a legacy data model with overlaysInformation and Software Technology10.1016/j.infsof.2009.04.01751:10(1454-1468)Online publication date: 1-Oct-2009
https://dl.acm.org/doi/10.1016/j.infsof.2009.04.017
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents