research-article

Who’s debugging the debuggers? exposing debug information bugs in optimized binaries

Authors:

Giuseppe Antonio Di Luna,

Davide Italiano,

Luca Massarelli,

Sebastian Österlund,

Cristiano Giuffrida,

Leonardo QuerzoniAuthors Info & Claims

ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

Pages 1034 - 1045

https://doi.org/10.1145/3445814.3446695

Published: 17 April 2021 Publication History

Abstract

Despite the advancements in software testing, bugs still plague deployed software and result in crashes in production. When debugging issues —sometimes caused by “heisenbugs”— there is the need to interpret core dumps and reproduce the issue offline on the same binary deployed. This requires the entire toolchain (compiler, linker, debugger) to correctly generate and use debug information. Little attention has been devoted to checking that such information is correctly preserved by modern toolchains’ optimization stages. This is particularly important as managing debug information in optimized production binaries is non-trivial, often leading to toolchain bugs that may hinder post-deployment debugging efforts.

In this paper, we present Debug², a framework to find debug information bugs in modern toolchains. Our framework feeds random source programs to the target toolchain and surgically compares the debugging behavior of their optimized/unoptimized binary variants. Such differential analysis allows Debug² to check invariants at each debugging step and detect bugs from invariant violations. Our invariants are based on the (in)consistency of common debug entities, such as source lines, stack frames, and function arguments. We show that, while simple, this strategy yields powerful cross-toolchain and cross-language invariants, which can pinpoint several bugs in modern toolchains. We have used Debug² to find 23 bugs in the LLVM toolchain (clang/lldb), 8 bugs in the GNU toolchain (GCC/gdb), and 3 in the Rust toolchain (rustc/lldb)—with 14 bugs already fixed by the developers.

References

[1]

Clang 12 documentation. https://clang.llvm.org/docs/CommandGuide/clang. html, 2020. [Online; accessed 27-July-2020].

[2]

Using-opt-bisect-limit to debug optimization errors. https://llvm.org/docs/ OptBisect.html, 2020. [Online; accessed 27-July-2020].

[3]

Ali-Reza Adl-Tabatabai and Thomas Gross. Source-level debugging of scalar optimized code. In Proceedings of the ACM SIGPLAN 1996 conference on Programming language design and implementation, pages 33-43, 1996.

Digital Library

[4]

Dmity Babokin, John Regehr, and Vsevolod Livinskiy. Yarpgen: Yet another random program generator. https://github.com/intel/yarpgen, 2020. [Online; accessed 27-July-2020].

[5]

Gary Brooks, Gilbert J Hansen, and Steve Simmons. A new approach to debugging optimized code. ACM SIGPLAN Notices, 27 ( 7 ): 1-11, 1992.

Digital Library

[6]

Junjie Chen, Jibesh Patra, Michael Pradel, Yingfei Xiong, Hongyu Zhang, Dan Hao, and Lu Zhang. A survey of compiler testing. ACM Computing Surveys (CSUR), 53 ( 1 ): 1-36, 2020.

[7]

Max Copperman. Debugging optimized code without being misled. ACM Transactions on Programming Languages and Systems (TOPLAS), 16 ( 3 ): 387-427, 1994.

[8]

Daniele Cono D'Elia and Camil Demetrescu. On-stack Replacement, Distilled. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 166-180. ACM, 2018.

Digital Library

[9]

V. D'Silva, M. Payer, and D. Song. The correctness-security gap in compiler optimization. In 2015 IEEE Security and Privacy Workshops, pages 73-87, 2015.

[10]

John Hennessy. Symbolic debugging of optimized code. ACM Transactions on Programming Languages and Systems (TOPLAS), 4 ( 3 ): 323-344, 1982.

[11]

Clara Jaramillo, Rajiv Gupta, and Mary Lou Sofa. Comparison checking: An approach to avoid debugging of optimized code. In Software EngineeringESEC/FSE'99, pages 268-284. Springer, 1999.

Digital Library

[12]

C. Jia and W. K. Chan. Which compiler optimization options should i use for detecting data races in multithreaded programs ? In 2013 8th International Workshop on Automation of Software Test (AST), pages 53-56, 2013.

[13]

Ken Kennedy and John R. Allen. Optimizing Compilers for Modern Architectures: A Dependence-Based Approach. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2001.

Digital Library

[14]

Vedant Kumar. How to update debug info: A guide for llvm pass authors. https://github.com/llvm/llvm-project/blob/master/llvm/docs/ HowToUpdateDebugInfo.rst, 2020. [Online; accessed 27-July-2020].

[15]

Vu Le, Mehrdad Afshari, and Zhendong Su. Compiler validation via equivalence modulo inputs. ACM SIGPLAN Notices, 49 ( 6 ): 216-226, 2014.

Digital Library

[16]

Daniel Lehmann and Michael Pradel. Feedback-directed diferential testing of interactive debuggers. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 610-620, 2018.

Digital Library

[17]

Xavier Leroy, Sandrine Blazy, Daniel Kästner, Bernhard Schommer, Markus Pister, and Christian Ferdinand. Compcert-a formally verified optimizing compiler. In ERTS 2016: Embedded Real Time Software and Systems. SEE, 2016.

[18]

Yuanbo Li, Shuo Ding, Qirun Zhang, and Davide Italiano. Debug information validation for optimized code. In PLDI, pages 1052-1065, 2020.

Digital Library

[19]

Duy Loc Phan, Yunho Kim, and Moonzoo Kim. Music: Mutation analysis tool with high configurability and extensibility. In 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 40-46. IEEE, 2018.

[20]

Adrian Prantl and Vedant Kumar. Debug info for optimized code. https://llvm.org/ devmtg/2018-10/slides/Prantl-Kumar-debug-info-bof-2018.pdf, 2020. [Online; accessed 27-July-2020].

[21]

John Regehr, Yang Chen, Pascal Cuoq, Eric Eide, Chucky Ellison, and Xuejun Yang. Test-case reduction for c compiler bugs. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ' 12, pages 335-346, New York, NY, USA, 2012. Association for Computing Machinery.

Digital Library

[22]

Eric Schulte, Jonathan Dorn, Stephen Harding, Stephanie Forrest, and Westley Weimer. Post-compiler software optimization for reducing energy. ACM SIGARCH Computer Architecture News, 42 ( 1 ): 639-652, 2014.

Digital Library

[23]

Chad Smith. pygdbmi-get structured output from gdb's machine interface. https://github.com/cs01/pygdbmi, 2020. [Online; accessed 27-July-2020].

[24]

Chengnian Sun, Vu Le, Qirun Zhang, and Zhendong Su. Toward understanding compiler bugs in gcc and llvm. In Proceedings of the 25th International Symposium on Software Testing and Analysis, pages 294-305, 2016.

Digital Library

[25]

Sandro Tolksdorf, Daniel Lehmann, and Michael Pradel. Interactive metamorphic testing of debuggers. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 273-283, 2019.

Digital Library

[26]

Roland Wismüller. Debugging of globally optimized programs using data flow analysis. In Proceedings of the ACM SIGPLAN 1994 conference on Programming language design and implementation, pages 278-289, 1994.

Digital Library

[27]

Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. Finding and understanding bugs in c compilers. In Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation, pages 283-294, 2011.

Digital Library

[28]

Jie Yin, Gang Tan, Hao Li, Xiaolong Bai, Yu-Ping Wang, and Shi-Min Hu. Debugopt: Debugging fully optimized natively compiled programs using multistage instrumentation. Science of Computer Programming, 169 : 18-32, 2019.

Cited By

Lu HLiu ZWang SZhang F(2024)DTD: Comprehensive and Scalable Testing for DebuggersProceedings of the ACM on Software Engineering10.1145/36437791:FSE(1172-1193)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643779
Metwalli SVan Meter R(2024)Testing and Debugging Quantum CircuitsIEEE Transactions on Quantum Engineering10.1109/TQE.2024.33748795(1-15)Online publication date: 2024
https://doi.org/10.1109/TQE.2024.3374879
Assaiante CNicchi SD’Elia DQuerzoni L(2024)Evading Userland API Hooking, Again: Novel Attacks and a Principled Defense MethodDetection of Intrusions and Malware, and Vulnerability Assessment10.1007/978-3-031-64171-8_8(150-173)Online publication date: 9-Jul-2024
https://doi.org/10.1007/978-3-031-64171-8_8
Show More Cited By

Index Terms

Who’s debugging the debuggers? exposing debug information bugs in optimized binaries
1. Software and its engineering

Recommendations

Debug information validation for optimized code
PLDI 2020: Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation

Almost all modern production software is compiled with optimization. Debugging optimized code is a desirable functionality. For example, developers usually perform post-mortem debugging on the coredumps produced by software crashes. Designing reliable ...
Compilation Consistency Modulo Debug Information
ASPLOS 2023: Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2

Compilation Consistency Modulo Debug Information (CCMD) is an essential compiler property that a production compiler should support: the compiler should emit the same machine code regardless of enabling debug information. CCMD is vital to developers’ ...
DTD: Comprehensive and Scalable Testing for Debuggers

As a powerful tool for developers, interactive debuggers help locate and fix errors in software. By using debugging information included in binaries, debuggers can retrieve necessary program states about the program. Unlike printf-style debugging, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASPLOS '21: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

April 2021

1090 pages

ISBN:9781450383172

DOI:10.1145/3445814

General Chair:
Tim Sherwood
University of California at Santa Barbara, USA
,
Program Chairs:
Emery Berger
University of Massachusetts at Amherst, USA
,
Christos Kozyrakis
Stanford University, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 April 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Funding Sources

H2020 Research and Innovation
AXA Research Fund
PNRM
Sapienza Università di Roma

Conference

ASPLOS '21

Sponsor:

SIGPLAN

ASPLOS '21: 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

April 19 - 23, 2021

Virtual, USA

Acceptance Rates

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
665
Total Downloads

Downloads (Last 12 months)96
Downloads (Last 6 weeks)16

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lu HLiu ZWang SZhang F(2024)DTD: Comprehensive and Scalable Testing for DebuggersProceedings of the ACM on Software Engineering10.1145/36437791:FSE(1172-1193)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643779
Metwalli SVan Meter R(2024)Testing and Debugging Quantum CircuitsIEEE Transactions on Quantum Engineering10.1109/TQE.2024.33748795(1-15)Online publication date: 2024
https://doi.org/10.1109/TQE.2024.3374879
Assaiante CNicchi SD’Elia DQuerzoni L(2024)Evading Userland API Hooking, Again: Novel Attacks and a Principled Defense MethodDetection of Intrusions and Malware, and Vulnerability Assessment10.1007/978-3-031-64171-8_8(150-173)Online publication date: 9-Jul-2024
https://doi.org/10.1007/978-3-031-64171-8_8
Isemann RGiuffrida CBos Hvan der Kouwe EGleissenthall K(2023)Don’t Look UB: Exposing Sanitizer-Eliding Compiler OptimizationsProceedings of the ACM on Programming Languages10.1145/35912577:PLDI(907-927)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1145/3591257
Brahmakshatriya AAmarasinghe SDubach CBruening DHardekopf B(2023)D2X: An eXtensible conteXtual Debugger for Modern DSLsProceedings of the 21st ACM/IEEE International Symposium on Code Generation and Optimization10.1145/3579990.3580014(162-172)Online publication date: 17-Feb-2023
https://dl.acm.org/doi/10.1145/3579990.3580014
Wang TTian YDong YXu ZSun CAamodt TJerger NSwift M(2023)Compilation Consistency Modulo Debug InformationProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3575693.3575740(146-158)Online publication date: 27-Jan-2023
https://dl.acm.org/doi/10.1145/3575693.3575740
Wang YZhang PSun MLu ZYang YTang YQian JLi ZZhou Y(2023)Uncovering Bugs in Code Coverage Profilers via Control Flow Constraint SolvingIEEE Transactions on Software Engineering10.1109/TSE.2023.332138149:11(4964-4987)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1109/TSE.2023.3321381
Hafkemeyer LStarink JContinella A(2023)Divak: Non-invasive Characterization of Out-of-Bounds Write VulnerabilitiesDetection of Intrusions and Malware, and Vulnerability Assessment10.1007/978-3-031-35504-2_11(211-232)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-35504-2_11
Kim GHong SFranz MSong DRyu SSmaragdakis Y(2022)Improving cross-platform binary analysis using representation learning via graph alignmentProceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3533767.3534383(151-163)Online publication date: 18-Jul-2022
https://dl.acm.org/doi/10.1145/3533767.3534383
Artuso FDi Luna GQuerzoni L(2022)Debugging Debug Information With Neural NetworksIEEE Access10.1109/ACCESS.2022.317661710(54136-54148)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3176617

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents