research-article

Simplifying and Isolating Failure-Inducing Input

Authors:

Andreas Zeller,

Ralf HildebrandtAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 28, Issue 2

Pages 183 - 200

https://doi.org/10.1109/32.988498

Published: 01 February 2002 Publication History

Publisher Site

Abstract

Given some test case, a program fails. Which circumstances of the test case are responsible for the particular failure? The Delta Debugging algorithm generalizes and simplifies the failing test case to a minimal test case that still produces the failure. It also isolates the difference between a passing and a failing test case. In a case study, the Mozilla web browser crashed after 95 user actions. Our prototype implementation automatically simplified the input to three relevant user actions. Likewise, it simplified 896 lines of HTML to the single line that caused the failure. The case study required 139 automated test runs or 35 minutes on a 500 MHz PC.

References

[1]

"Mozilla web site,"http://www.mozilla.org/, 2001.

Google Scholar

[2]

"Mozilla Web Site: The Gecko BugAThon,"http:!/www.mozilla. org/new layout !bugathon.html. 2001.

Google Scholar

[3]

A Zeller, Yesterday, My Program Worked. Today, It Does Not. Why? Proc. Seventh European Software Eng. Conf, Seventh ACM SIGSOFT Syinp. Foundations of Software Eng., (ESEC/ESE '99), 0. Nierstrasz and M. Lemoine, eds. vol. 1687, pp. 253-267, Sept. 1999.

Digital Library

Google Scholar

[4]

Test Methods for Measuring Conformance to POSIX. 1991, ANSI! IEEE Standard 1003.31991. ISO/IEC Standard 13210, 1994.

Google Scholar

[5]

M. Vertes, "Male-A Tool to Automate Graphical User Interfaces Linux Weekly News, May 1998, Archived as http://lwn.net/ 980528/a/ xlab.html.

Google Scholar

[6]

B.P. Miller, L Fredrikson, and B. So, "An Empirical Study of the Reliability of UNIX Utilities," Comm. ACM, vol. 33, no, 12, pp. 32-44, Dec. 1990.

Digital Library

Google Scholar

[7]

B.P. Miller, D. Koski, C.P. Lee, V. Maganty, R. Murthy, A. Natarajan, and J. Steidi, "Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and Services," technical report, Univ. of Wisconsin, Computer Science Dept., Nov. 1995.

Google Scholar

[8]

D.R. Slutz, "Massive Stochastic Testing of SQL," Proc. 24th Int'l Conf. Very Large Data Bases (VLDB '98), A. Gupta, 0. Shmueli, and J, Widom, eds., pp. 618-622, Aug. 1998.

Digital Library

Google Scholar

[9]

D.B. Whalley, "Automatic Isolation of Compiler Errors," ACM Trans. Programming Languages and Systems, vol. 16, no. 5, pp. 1648-1659, 1994.

Digital Library

Google Scholar

[10]

A. Zeller and G. Snelting, "Unified Versioning through Feature Logic," ACM Trans. Software Eng. and Methodology, vol. 6, no. 4, pp. 398-441, Oct. 1997.

Digital Library

Google Scholar

[11]

M. Weiser, "Programmers Use Slices when Debugging," Comm. ACM, vol. 25. no. 7, pp. 446-452, 1982.

Digital Library

Google Scholar

[12]

F. Tip, "A Survey of Program Slicing Techniques," /. Programming Languages, vol. 3, no. 3, pp. 121-189, Sept. 1995.

Google Scholar

[13]

H. Agrawal and JR. Horgan, "Dynamic Program Slicing," Proc. ACM SIGPLAN 1990 Conf Programming Language Design and Implementation (PLDI), vol. 25, no. 6, pp. 246-256, June 1990.

Digital Library

Google Scholar

[14]

t. GyimOthy, A. Beszedes, and 1. Forgcs, "An Efficient Relevant Slicing Method for Debugging," Proc. Seventh European Software big Conf, Seventh ACM SIGSOFT Symp. Foundations of Software Log, (ESEC/FSE '99), 0. Nierstrasz and M. Lemoine, eds vol. 1687, pp. 303-321, Sept. 1999.

Digital Library

Google Scholar

Cited By

View all

Padhye REdwards JTaeumel M(2024)Software Engineering Methods for AI-Driven Deductive Legal ReasoningProceedings of the 2024 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3689492.3690050(85-95)Online publication date: 17-Oct-2024
https://dl.acm.org/doi/10.1145/3689492.3690050
van den Broek SOprescu ARuys TGonzalez Boix EScholliers C(2024)Bug Localisation in Model-Based Testing using Model-Centric ShrinkingProceedings of the 2nd ACM International Workshop on Future Debugging Techniques10.1145/3678720.3685315(9-15)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678720.3685315
Vince DKiss ÁPrasetya WVos TStorm T(2024)GreeDDy: Accelerate Parallel DDMINProceedings of the 15th ACM International Workshop on Automating Test Case Design, Selection and Evaluation10.1145/3678719.3685690(1-4)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678719.3685690
Show More Cited By

Index Terms

Simplifying and Isolating Failure-Inducing Input
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Simplifying failure-inducing input

Given some test case, a program fails. Which part of the test case is responsible for the particular failure? We show how our delta debugging algorithm generalizes and simplifies some failing input to a minimal test case that produces the failure.
...
Simplifying failure-inducing input
ISSTA '00: Proceedings of the 2000 ACM SIGSOFT international symposium on Software testing and analysis

Given some test case, a program fails. Which part of the test case is responsible for the particular failure? We show how our delta debugging algorithm generalizes and simplifies some failing input to a minimal test case that produces the failure.
...
Isolating Failure-Inducing Combinations in Combinatorial Testing Using Test Augmentation and Classification
ICST '12: Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation

Combinatorial Testing (CT) is a systematic way of sampling input parameters of the software under test (SUT). A t-way combinatorial test set can exercise all behaviors of the SUT caused by interactions between t input parameters or less. Although ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 28, Issue 2

February 2002

105 pages

ISSN:0098-5589

Issue’s Table of Contents

Publisher

IEEE Press

Publication History

Published: 01 February 2002

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

482
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Padhye REdwards JTaeumel M(2024)Software Engineering Methods for AI-Driven Deductive Legal ReasoningProceedings of the 2024 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3689492.3690050(85-95)Online publication date: 17-Oct-2024
https://dl.acm.org/doi/10.1145/3689492.3690050
van den Broek SOprescu ARuys TGonzalez Boix EScholliers C(2024)Bug Localisation in Model-Based Testing using Model-Centric ShrinkingProceedings of the 2nd ACM International Workshop on Future Debugging Techniques10.1145/3678720.3685315(9-15)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678720.3685315
Vince DKiss ÁPrasetya WVos TStorm T(2024)GreeDDy: Accelerate Parallel DDMINProceedings of the 15th ACM International Workshop on Automating Test Case Design, Selection and Evaluation10.1145/3678719.3685690(1-4)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678719.3685690
Qiu YHu JZhang QYin HChristakis MPradel M(2024)Calico: Automated Knowledge Calibration and Diagnosis for Elevating AI Mastery in Code TasksProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680399(1785-1797)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680399
Tan JRigger MChristakis MPradel M(2024)Inconsistencies in TeX-Produced DocumentsProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680370(1415-1427)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680370
Ma HZhang WShen QTian YChen JCheung SChristakis MPradel M(2024)Towards Understanding the Bugs in Solidity CompilerProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680362(1312-1324)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680362
Lin LHao ZWang CWang ZWu RFan GChristakis MPradel M(2024)SQLess: Dialect-Agnostic SQL Query SimplificationProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680317(743-754)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680317
Chu ZWan YLi QWu YZhang HSui YXu GJin HChristakis MPradel M(2024)Graph Neural Networks for Vulnerability Detection: A Counterfactual ExplanationProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652136(389-401)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652136
Song XWu YLiu SChen BLin YPeng XChristakis MPradel M(2024)C2D2: Extracting Critical Changes for Real-World Bugs with Dependency-Sensitive Delta DebuggingProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652129(300-312)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652129
Zhang MTian YXu ZDong YTan SSun CChristakis MPradel M(2024)LPR: Large Language Models-Aided Program ReductionProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652126(261-273)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652126
Show More Cited By

Abstract

References

Cited By

Index Terms

Recommendations

Simplifying failure-inducing input

Simplifying failure-inducing input

Isolating Failure-Inducing Combinations in Combinatorial Testing Using Test Augmentation and Classification

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations