Cited By
View all- Ren LZhang XHua ZJiang YHe XXiong YXie T(2024)Validity-Preserving Delta Debugging via Generator Trace ReductionACM Transactions on Software Engineering and Methodology10.1145/3705305Online publication date: 23-Dec-2024
T-Rec: Fine-Grained Language-Agnostic Program Reduction Guided by Lexical Syntax
Authors: 
Zhenyang Xu, Yongqiang Tian, Mengxiao Zhang, Jiarui Zhang, 
Puzhuo Liu, Yu Jiang, Chengnian SunAuthors Info & Claims
Zhenyang Xu, Yongqiang Tian, Mengxiao Zhang, Jiarui Zhang, Puzhuo Liu, Yu Jiang, Chengnian SunAuthors Info & ClaimsArticle No.: 34, Pages 1 - 31
Abstract
Program reduction strives to eliminate bug-irrelevant code elements from a bug-triggering program, so that (1) a smaller and more straightforward bug-triggering program can be obtained, (2) and the difference among duplicates (i.e., different programs that trigger the same bug) can be minimized or even eliminated. With such reduction and canonicalization functionality, program reduction facilitates debugging for software, especially language toolchains, such as compilers, interpreters, and debuggers. While many program reduction techniques have been proposed, most of them (especially the language-agnostic ones) overlooked the potential reduction opportunities hidden within tokens. Therefore, their capabilities in terms of reduction and canonicalization are significantly restricted.
To fill this gap, we propose \(\mathsf{T}\)-\(\mathsf{Rec}\), a fine-grained language-agnostic program reduction technique guided by lexical syntax. Instead of treating tokens as atomic and irreducible components, \(\mathsf{T}\)-\(\mathsf{Rec}\) introduces a fine-grained reduction process that leverages the lexical syntax of programming languages to effectively explore the reduction opportunities in tokens. Through comprehensive evaluations with versatile benchmark suites, we demonstrate that \(\mathsf{T}\)-\(\mathsf{Rec}\) significantly improves the reduction and canonicalization capability of two existing language-agnostic program reducers (i.e., Perses and Vulcan). \(\mathsf{T}\)-\(\mathsf{Rec}\) enables Perses and Vulcan to further eliminate 1,294 and 1,315 duplicates in a benchmark suite that contains 3,796 test cases that trigger 46 unique bugs. Additionally, \(\mathsf{T}\)-\(\mathsf{Rec}\) can also reduce up to 65.52% and 53.73% bytes in the results of Perses and Vulcan on our multi-lingual benchmark suite, respectively.
References
[1]
ANTLR. 2017. The ANTLR Parser Generator. Retrieved September 20, 2022 from https://www.antlr.org/
[2]
Cornelius Aschermann, Tommaso Frassetto, Thorsten Holz, Patrick Jauernig, Ahmad-Reza Sadeghi, and Daniel Teuchert. 2019. NAUTILUS: Fishing for deep bugs with grammars. In Proceedings of the 26th Annual Network and Distributed System Security Symposium (NDSS ’19). The Internet Society. Retrieved from https://www.ndss-symposium.org/ndss-paper/nautilus-fishing-for-deep-bugs-with-grammars/
[3]
Yang Chen, Alex Groce, Chaoqiang Zhang, Weng-Keen Wong, Xiaoli Fern, Eric Eide, and John Regehr. 2013. Taming compiler fuzzers. In Proceedings of the 2013 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’13), 197–208.
[4]
Nathan Chong, Alastair Donaldson, Andrei Lascu, and Christopher Lidbury. 2015. Many-core compiler fuzzing. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15), 65–76.
[5]
CPython. 2022. Bug Report. Retrieved September 20, 2022 from https://github.com/python/cpython/issues/new?assignees={&}labels=type-bug{&}template=bug.md
[6]
Alastair Donaldson and David MacIver. 2021. Test Case Reduction: Beyond Bugs. Retrieved May 29, 2023 from https://blog.sigplan.org/2021/05/25/test-case-reduction-beyond-bugs
[7]
GCC. 2020. A Guide to Testcase Reduction. Retrieved October 28, 2022 from https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction
[8]
GCC-Wiki. 2020. A Guide to Testcase Reduction. Retrieved September 20, 2022 from https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction
[9]
Golnaz Gharachorlu and Nick Sumner. 2023. Type batched program reduction. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, 398–410.
[10]
Rahul Gopinath, Alexander Kampmann, Nikolas Havrikov, Ezekiel O. Soremekun, and Andreas Zeller. 2020a. Abstracting failure-inducing inputs. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, 237–248.
[11]
Rahul Gopinath, Alexander Kampmann, Nikolas Havrikov, Ezekiel O. Soremekun, and Andreas Zeller. 2020b. The DDSET Github Repo. Retrieved June 20, 2024 from https://github.com/vrthra/ddset
[12]
Alex Groce, Josie Holmes, and Kevin Kellar. 2017. One test to rule them all. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’17). ACM, New York, NY, USA, 1–11. DOI:
[13]
Kihong Heo, Woosuk Lee, Pardis Pashakhanloo, and Mayur Naik. 2018. Effective program debloating via reinforcement learning. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS ’18). David Lie, Mohammad Mannan, Michael Backes, and XiaoFeng Wang (Eds.), ACM, 380–394. DOI:
[14]
Renáta Hodován and Ákos Kiss. 2016. Modernizing hierarchical delta debugging. In Proceedings of the 7th International Workshop on Automating Test Case Design, Selection, and Evaluation (A-TEST ’16). ACM, New York, NY, USA, 31–37. DOI:
[15]
Renáta Hodován, Ákos Kiss, and Tibor Gyimóthy. 2017. Coarse hierarchical delta debugging. In Proceedings of the 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME ’17). IEEE Computer Society, 194–203. DOI:
[16]
Renáta Hodován, Ákos Kiss, and Tibor Gyimóthy. 2018. Grammarinator: A grammar-based open source fuzzer. In Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation (A-TEST ’18). ACM, New York, NY, USA, 45–48. DOI:
[17]
JerryScript. 2022. Bug Report. Retrieved September 20, 2022 from https://github.com/jerryscript-project/jerryscript/blob/master/.github/ISSUE_TEMPLATE/bug_report.md
[18]
JS Delta. 2017. JS Delta. Retrieved October 28, 2022 from https://github.com/wala/jsdelta
[19]
Christian Gram Kalhauge and Jens Palsberg. 2019. Binary reduction of dependency graphs. In Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/SIGSOFT FSE ’19). Marlon Dumas, Dietmar Pfahl, Sven Apel, and Alessandra Russo (Eds.), ACM, 556–566. DOI:
[20]
Christian Gram Kalhauge and Jens Palsberg. 2021. Logical bytecode reduction. In Proceedings of the PLDI ’21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation. Stephen N. Freund and Eran Yahav (Eds.), ACM, 1003–1016. DOI:
[21]
Ákos Kiss, Renáta Hodován, and Tibor Gyimóthy. 2018. HDDr: A recursive variant of the hierarchical delta debugging algorithm. In Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation (A-TEST ’18). ACM, New York, NY, USA, 16–22. DOI:
[22]
Gereon Kremer, Aina Niemetz, and Mathias Preiner. 2021. ddSMT 2.0: Better delta debugging for the SMT-LIBv2 language and friends. In Proceedings of the Computer Aided Verification - 33rd International Conference (CAV ’21). Alexandra Silva and K. Rustan M. Leino (Eds.), Part II (Lecture Notes in Computer Science, Vol. 12760), Springer, 231–242. DOI:
[23]
Vu Le, Mehrdad Afshari, and Zhendong Su. 2014. Compiler validation via equivalence modulo inputs. 216–226. DOI:
[24]
Cong Li, Yanyan Jiang, Chang Xu, and Zhendong Su. 2023. Validating JIT compilers via compilation space exploration. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP ’23). ACM, New York, NY, USA, 66–79. DOI:
[25]
Vsevolod Livinskii, Dmitry Babokin, and John Regehr. 2020. Random testing for C and C++ compilers with YARPGen. Proceedings of the ACM on Programming Languages 4, OOPSLA (2020), 1–25.
[26]
LLVM. 2022. How to Submit an LLVM Bug Report. Retrieved September 20, 2022 from https://llvm.org/docs/HowToSubmitABug.html
[27]
David R. MacIver and Alastair F. Donaldson. 2020. Test-case reduction via test-case generation: Insights from the hypothesis reducer (tool insights paper). In Proceedings of the 34th European Conference on Object-Oriented Programming (ECOOP ’20). Schloss-Dagstuhl-Leibniz Zentrum für Informatik.
[28]
Ghassan Misherghi and Zhendong Su. 2006. HDD: Hierarchical Delta Debugging. In Proceedings of the 28th International Conference on Software Engineering (ICSE ’06). Leon J. Osterweil, H. Dieter Rombach, and Mary Lou Soffa (Eds.), ACM, 142–151. DOI:
[29]
Aina Niemetz and Armin Biere. 2013. ddSMT: A delta debugger for the SMT-LIB v2 format. In Proceedings of the 11th International Workshop on Satisfiability Modulo Theories, SMT, 8–9.
[30]
John Regehr. 2019. Design and Evolution of C-Reduce. Retrieved January 20, 2024 from https://blog.regehr.org/archives/1679
[31]
John Regehr, Yang Chen, Pascal Cuoq, Eric Eide, Chucky Ellison, and Xuejun Yang. 2012. Test-case reduction for C compiler bugs. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’12). ACM, New York, NY, USA, 335–346. DOI:
[32]
Manuel Rigger and Zhendong Su. 2020. Detecting optimization bugs in database engines via non-optimizing reference engine construction. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’20). ACM, New York, NY, USA, 1140–1152. DOI:
[33]
Rust. 2024. Finding a Minimal, Standalone Example. Retrieved Januray 20, 2024 from https://rustc-dev-guide.rust-lang.org/notification-groups/cleanup-crew.html#finding-a-minimal-standalone-example
[34]
Prashast Srivastava and Mathias Payer. 2021. Gramatron: Effective grammar-aware fuzzing. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, 244–256.
[35]
Chengnian Sun, Vu Le, and Zhendong Su. 2016. Finding compiler bugs via live code mutation. In Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA ’16, part of SPLASH ’16). Eelco Visser and Yannis Smaragdakis (Eds.), ACM, 849–863. DOI:
[36]
Chengnian Sun, Yuanbo Li, Qirun Zhang, Tianxiao Gu, and Zhendong Su. 2018. Perses: Syntax-guided program reduction. In Proceedings of the 40th International Conference on Software Engineering (ICSE ’18). ACM, New York, NY, USA, 361–371. DOI:
[37]
Yongqiang Tian, Xueyan Zhang, Yiwen Dong, Zhenyang Xu, Mengxiao Zhang, Yu Jiang, Shing-Chi Cheung, and Chengnian Sun. 2023. On the caching schemes to speed up program reduction. ACM Transactions on Software Engineering and Methodology 33, 1, Article 17 (Nov 2023), 30 pages. DOI:
[38]
Dániel Vince. 2022. Iterating the minimizing Delta debugging algorithm. In Proceedings of the 13th International Workshop on Automating Test Case Design, Selection and Evaluation (A-TEST ’22). Ákos Kiss, Beatriz Marín, and Mehrdad Saadatmand (Eds.), ACM, 57–60. DOI:
[39]
Dániel Vince, Renáta Hodován, Daniella Bársony, and Ákos Kiss. 2021. Extending hierarchical delta debugging with hoisting. In Proceedings of the 2021 IEEE/ACM International Conference on Automation of Software Test (AST). IEEE, 60–69.
[40]
Guancheng Wang, Ruobing Shen, Junjie Chen, Yingfei Xiong, and Lu Zhang. 2021. Probabilistic Delta debugging. In Proceedings of the ESEC/FSE ’21: 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. Diomidis Spinellis, Georgios Gousios, Marsha Chechik, and Massimiliano Di Penta (Eds.), ACM, 881–892. DOI:
[41]
Frank Wilcoxon. 1992. Individual Comparisons by Ranking Methods. Springer.
[42]
Dominik Winterer, Chengyu Zhang, and Zhendong Su. 2020. Validating SMT solvers via semantic fusion. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’20). ACM, New York, NY, USA, 718–730. DOI:
[43]
Zhenyang Xu, Yongqiang Tian, Mengxiao Zhang, Gaosen Zhao, Yu Jiang, and Chengnian Sun. 2023. Pushing the limit of 1-minimality of language-agnostic program reduction. Proceedings of the ACM on Programming Languages 7, OOPSLA1, Article 97 (Apr 2023), 29 pages. DOI:
[44]
Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. 2011. Finding and understanding bugs in C compilers. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’11). ACM, New York, NY, USA, 283–294. DOI:
[45]
A. Zeller and R. Hildebrandt. 2002. Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering 28, 2 (2002), 183–200. DOI:
[46]
Mengxiao Zhang, Yongqiang Tian, Zhenyang Xu, Yiwen Dong, Shin Hwei Tan, and Chengnian Sun. 2024. LPR: Large language models-aided program reduction. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, 261–273.
[47]
Mengxiao Zhang, Zhenyang Xu, Yongqiang Tian, Yu Jiang, and Chengnian Sun. 2023. PPR: Pairwise program reduction. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 338–349.
Index Terms
- T-Rec: Fine-Grained Language-Agnostic Program Reduction Guided by Lexical Syntax
Recommendations
LPR: Large Language Models-Aided Program Reduction
ISSTA 2024: Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and AnalysisProgram reduction is a widely used technique to facilitate debugging compilers by automatically minimizing programs that trigger compiler bugs. Existing program reduction techniques are either generic to a wide range of languages (such as Perses and ...
Perses: syntax-guided program reduction
ICSE '18: Proceedings of the 40th International Conference on Software EngineeringGiven a program P that exhibits a certain property Ψ (e.g., a C program that crashes GCC when it is being compiled), the goal of program reduction is to minimize P to a smaller variant P′ that still exhibits the same property, i.e., Ψ(P′). Program ...
PPR: Pairwise Program Reduction
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software EngineeringProgram reduction is a practical technique widely used for debugging compilers. To report a compiler bug with a bug-triggering program, one needs to minimize the program by removing bugirrelevant program elements first. Though existing program reduction ...
Comments
Information & Contributors
Information
Published In
February 2025
904 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 January 2025
Online AM: 30 August 2024
Accepted: 10 August 2024
Revised: 05 July 2024
Received: 08 February 2024
Published in TOSEM Volume 34, Issue 2
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Natural Sciences and Engineering Research Council of Canada (NSERC)
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 164Total Downloads
- Downloads (Last 12 months)164
- Downloads (Last 6 weeks)53
Reflects downloads up to 22 Jan 2025
Other Metrics
Citations
Cited By
View all- Ren LZhang XHua ZJiang YHe XXiong YXie T(2024)Validity-Preserving Delta Debugging via Generator Trace ReductionACM Transactions on Software Engineering and Methodology10.1145/3705305Online publication date: 23-Dec-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in