Article

An empirical study of dormant bugs

Authors:

Meiyappan Nagappan,

Ahmed E. HassanAuthors Info & Claims

MSR 2014: Proceedings of the 11th Working Conference on Mining Software Repositories

Pages 82 - 91

https://doi.org/10.1145/2597073.2597108

Published: 31 May 2014 Publication History

Abstract

Over the past decade, several research efforts have studied the quality of software systems by looking at post-release bugs. However, these studies do not account for bugs that remain dormant (i.e., introduced in a version of the software system, but are not found until much later) for years and across many versions. Such dormant bugs skew our under- standing of the software quality. In this paper we study dormant bugs against non-dormant bugs using data from 20 different open-source Apache foundation software systems. We find that 33% of the bugs introduced in a version are not reported till much later (i.e., they are reported in future versions as dormant bugs). Moreover, we find that 18.9% of the reported bugs in a version are not even introduced in that version (i.e., they are dormant bugs from prior versions). In short, the use of reported bugs to judge the quality of a specific version might be misleading. Exploring the fix process for dormant bugs, we find that they are fixed faster (median fix time of 5 days) than non- dormant bugs (median fix time of 8 days), and are fixed by more experienced developers (median commit counts of developers who fix dormant bug is 169% higher). Our results highlight that dormant bugs are different from non-dormant bugs in many perspectives and that future research in software quality should carefully study and consider dormant bugs.

References

[1]

Nachiappan Nagappan and Thomas Ball. Use of relative code churn measures to predict system defect density. In Proceedings of the 27th international conference on Software engineering, ICSE ’05, pages 284–292, 2005.

Digital Library

[2]

Thomas Zimmermann, Rahul Premraj, and Andreas Zeller. Predicting defects for eclipse. In Proceedings of the Third International Workshop on Predictor Models in Software Engineering, PROMISE ’07, 2007.

Digital Library

[3]

Ahmed E. Hassan. Predicting faults using the complexity of code changes. In ICSE ’09, pages 78–88, 2009.

Digital Library

[4]

Nachiappan Nagappan, Thomas Ball, and Andreas Zeller. Mining metrics to predict component failures. In ICSE ’06, pages 452–461, 2006.

Digital Library

[5]

Emad Shihab, Audris Mockus, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan. High-impact defects: a study of breakage and surprise defects. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, ESEC/FSE ’11, pages 300–310, 2011.

Digital Library

[6]

Emad Shihab, Zhen Ming Jiang, Walid M. Ibrahim, Bram Adams, and Ahmed E. Hassan. Understanding the impact of code and process metrics on post-release defects: a case study on the eclipse project. In Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM ’10, 2010.

Digital Library

[7]

Tse-Hsun Chen, S. W. Thomas, Meiyappan Nagappan, and A.E. Hassan. Explaining software defects using topic models. In Proceedings of the 9th Working Conference on Mining Software Repositories, MSR ’12, 2012.

Digital Library

[8]

Thomas Zimmermann, Nachiappan Nagappan, Harald Gall, Emanuel Giger, and Brendan Murphy. Cross-project defect prediction: a large scale experiment on data vs. domain vs. process. In ESEC/FSE ’09, 2009.

Digital Library

[9]

Adrian Schrˇ Zter, Thomas Zimmermann, Rahul Premraj, and Andreas Zeller. If your bug database could talk. In Proceedings of the 5th International Symposium on Empirical Software Engineering, Volume II: Short Papers and Posters, pages 18–20, 2006.

[10]

Guoliang Jin, Linhai Song, Xiaoming Shi, Joel Scherpelz, and Shan Lu. Understanding and detecting real-world performance bugs. In Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation, PLDI ’12, 2012.

Digital Library

[11]

Shan Lu, Zhenmin Li, Feng Qin, Lin Tan, Pin Zhou, and Yuanyuan Zhou. Bugbench: Benchmarks for evaluating bug detection tools. In Workshop on the Evaluation of Software Defect Detection Tools, 2005.

[12]

Christian Bird, Nachiappan Nagappan, Brendan Murphy, Harald Gall, and Premkumar Devanbu. Don’t touch my code!: Examining the effects of ownership on software quality. In SIGSOFT/FSE ’11, pages 4–14, 2011.

Digital Library

[13]

Nachiappan Nagappan and Thomas Ball. Static analysis tools as early indicators of pre-release defect density. In ICSE ’05, pages 580–586, 2005.

Digital Library

[14]

Thomas Zimmermann, Rahul Premraj, and Andreas Zeller. Predicting defects for Eclipse. In PROMISE ’07: Proceedings of the Third International Workshop on Predictor Models in Software Engineering, page 9, 2007.

Digital Library

[15]

F. Khomh, T. Dhaliwal, Ying Zou, and B. Adams. Do faster releases improve software quality? an empirical case study of mozilla firefox. In Proceedings of the 9th International Working Conference on Mining Software Repositories, pages 179–188, June 2012.

[16]

Thomas Zimmermann, Nachiappan Nagappan, and Laurie Williams. Searching for a needle in a haystack: Predicting security vulnerabilities for windows vista. In Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation, ICST ’10, pages 421–428, 2010.

Digital Library

[17]

Zhenmin Li, Lin Tan, Xuanhui Wang, Shan Lu, Yuanyuan Zhou, and Chengxiang Zhai. Have things changed now?: an empirical study of bug characteristics in modern open source software. In Proceedings of the 1st workshop on Architectural and system support for improving software dependability, 2006.

Digital Library

[18]

Adrian Nistor, Tian Jiang, and Lin Tan. Discovering, reporting, and fixing performance bugs. In MSR ’13, May 2013.

Digital Library

[19]

Shahed Zaman, Bram Adams, and Ahmed E. Hassan. Security versus performance bugs: a case study on firefox. In MSR ’11, pages 93–102, 2011.

Digital Library

[20]

Apache Software Foundation. Jira. https://issues.apache.org/jira/, 2013.

[21]

Sunghun Kim, Hongyu Zhang, Rongxin Wu, and Liang Gong. Dealing with noise in defect prediction. In ICSE ’11, pages 481–490, 2011.

Digital Library

[22]

Foyzur Rahman, Daryl Posnett, and Premkumar Devanbu. Recalling the ”imprecision” of cross-project defect prediction. In SIGSOFT/FSE ’11, FSE ’12, 2012.

Digital Library

[23]

Daryl Posnett, Abram Hindle, and Prem Devanbu. Got issues? do new features and code improvements affect defects? In WCRE ’11, pages 211–215, 2011.

Digital Library

[24]

Foyzur Rahman, Daryl Posnett, Israel Herraiz, and Premkumar Devanbu. Sample size vs. bias in defect prediction. In ESEC/FSE 2013, pages 147–157, 2013.

Digital Library

[25]

Audris Mockus, Roy T. Fielding, and James D. Herbsleb. Two case studies of open source software development: Apache and mozilla. ACM Trans. Softw. Eng. Methodol., 11(3):309–346, July 2002.

Digital Library

[26]

Gaeul Jeong, Sunghun Kim, and Thomas Zimmermann. Improving bug triage with bug tossing graphs. In SIGSOFT/FSE ’09, ESEC/FSE ’09, pages 111–120, 2009.

Digital Library

[27]

Zuoning Yin, Ding Yuan, Yuanyuan Zhou, Shankar Pasupathy, and Lakshmi Bairavasundaram. How do fixes become bugs? In ESEC/FSE ’11, pages 26–36, 2011.

Digital Library

[28]

Emad Shihab, Akinori Ihara, Yasutaka Kamei, Walid M. Ibrahim, Masao Ohira, Bram Adams, Ahmed E. Hassan, and Ken-ichi Matsumoto. Studying re-opened bugs in open source software. Empirical Software Engineering, 2012.

[29]

Audris Mockus and James D. Herbsleb. Expertise browser: a quantitative approach to identifying expertise. In ICSE ’02, pages 503–512, 2002.

Digital Library

[30]

H. Kagdi, M. Hammad, and J.I. Maletic. Who can help me with this source code change? In Proceedings of the 24th International Conference on Software Maintenance, ICSM ’08, pages 157–166, 2008.

[31]

Foyzur Rahman and Premkumar Devanbu. Ownership, experience and defects: A fine-grained study of authorship. In ICSE ’11, pages 491–500, 2011.

Digital Library

[32]

Ran Tang, Ahmed E. Hassan, and Ying Zou. Techniques for identifying the country origin of mailing list participants. In Proceedings of the 2009 16th Working Conference on Reverse Engineering, WCRE ’09, pages 36–40, 2009.

Digital Library

[33]

S. Boslaugh and P.A. Watters. Statistics in a Nutshell: A Desktop Quick Reference. In a Nutshell (O’Reilly). O’Reilly Media, 2008.

Digital Library

[34]

Apache Derby. Derby-1661. https: //issues.apache.org/jira/browse/DERBY-1661, 2013.

[35]

Apache Wicket. Wicket-4161. 2013.

[36]

Apache Felix. Felix-1460. https: //issues.apache.org/jira/browse/FELIX-1460, 2013.

[37]

Emad Shihab. An Exploration of Challenges Limiting Pragmatic Software Defect Prediction. PhD thesis, 2012.

[38]

Jacek Śliwerski, Thomas Zimmermann, and Andreas Zeller. When do changes induce fixes? In MSR ’05, pages 1–5, 2005.

Digital Library

[39]

Vigdis By Kampenes, Tore Dyb˚ a, Jo E. Hannay, and Dag I. K. Sjøberg. Systematic review: A systematic review of effect size in software engineering experiments. Inf. Softw. Technol., 49(11-12):1073–1086, November 2007.

Digital Library

Cited By

Jiang MJiang JWu TMa ZLuo XZhou Y(2024)Understanding Vulnerability Inducing Commits of the Linux KernelACM Transactions on Software Engineering and Methodology10.1145/367245233:7(1-28)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672452
Tamanna SUddin GXia LZhang LOnut IShirani POnut IOnut IBranco P(2023)Characterizing Issue Management in Runtime SystemsProceedings of the 33rd Annual International Conference on Computer Science and Software Engineering10.5555/3615924.3615930(54-63)Online publication date: 11-Sep-2023
https://dl.acm.org/doi/10.5555/3615924.3615930
Barbosa KFerreira RPinto Gd'Amorim MMiranda B(2023)Test Flakiness Across Programming LanguagesIEEE Transactions on Software Engineering10.1109/TSE.2022.320886449:4(2039-2052)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TSE.2022.3208864
Show More Cited By

Index Terms

An empirical study of dormant bugs
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Project and people management
2. Software and its engineering
  1. Software creation and management
    1. Software development process management
  2. Software notations and tools
    1. Software configuration management and version control systems

Recommendations

Security versus performance bugs: a case study on Firefox
MSR '11: Proceedings of the 8th Working Conference on Mining Software Repositories

A good understanding of the impact of different types of bugs on various project aspects is essential to improve software quality research and practice. For instance, we would expect that security bugs are fixed faster than other types of bugs due to ...
An empirical study of bugs in build process
SAC '14: Proceedings of the 29th Annual ACM Symposium on Applied Computing

Software build process translates source codes into executable programs, packages the programs, generates documents, and distributes products. In this paper, we perform an empirical study to characterize build process bugs. We analyze bugs in build ...
An empirical study on bugs in JavaScript engines
Abstract Context:
JavaScript is a prototype-based dynamic type scripting language. The correct running of a JavaScript program depends on the correctness of both the program and the JavaScript engine.
Objective: ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MSR 2014: Proceedings of the 11th Working Conference on Mining Software Repositories

May 2014

427 pages

ISBN:9781450328630

DOI:10.1145/2597073

General Chair:
Premkumar Devanbu
University of California at Davis, USA
,
Program Chairs:
Sung Kim
Hong Kong University of Science and Technology, China
,
Martin Pinzger
University of Klagenfurt, Austria

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICSE '14

Sponsor:

SIGSOFT

ICSE '14: 36th International Conference on Software Engineering

May 31 - June 1, 2014

Hyderabad, India

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

62
Total Citations
View Citations
535
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)4

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jiang MJiang JWu TMa ZLuo XZhou Y(2024)Understanding Vulnerability Inducing Commits of the Linux KernelACM Transactions on Software Engineering and Methodology10.1145/367245233:7(1-28)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672452
Tamanna SUddin GXia LZhang LOnut IShirani POnut IOnut IBranco P(2023)Characterizing Issue Management in Runtime SystemsProceedings of the 33rd Annual International Conference on Computer Science and Software Engineering10.5555/3615924.3615930(54-63)Online publication date: 11-Sep-2023
https://dl.acm.org/doi/10.5555/3615924.3615930
Barbosa KFerreira RPinto Gd'Amorim MMiranda B(2023)Test Flakiness Across Programming LanguagesIEEE Transactions on Software Engineering10.1109/TSE.2022.320886449:4(2039-2052)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TSE.2022.3208864
Song LMinku L(2023)A Procedure to Continuously Evaluate Predictive Performance of Just-In-Time Software Defect Prediction Models During Software DevelopmentIEEE Transactions on Software Engineering10.1109/TSE.2022.315883149:2(646-666)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TSE.2022.3158831
Liu SGuo ZLi YWang CChen LSun ZZhou YXu B(2023)Inconsistent Defect Labels: Essence, Causes, and InfluenceIEEE Transactions on Software Engineering10.1109/TSE.2022.315678749:2(586-610)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TSE.2022.3156787
Esposito MFalessi D(2023)Uncovering the Hidden Risks: The Importance of Predicting Bugginess in Untouched Methods2023 IEEE 23rd International Working Conference on Source Code Analysis and Manipulation (SCAM)10.1109/SCAM59687.2023.00039(277-282)Online publication date: 2-Oct-2023
https://doi.org/10.1109/SCAM59687.2023.00039
Xia XFeng YShi Q(2023)Understanding Bugs in Rust Compilers2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security (QRS)10.1109/QRS60937.2023.00023(138-149)Online publication date: 22-Oct-2023
https://doi.org/10.1109/QRS60937.2023.00023
Xiao XXiao RLi QLv JCui SLiu Q(2023)BugRadarInformation and Software Technology10.1016/j.infsof.2023.107274162:COnline publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.infsof.2023.107274
Song LMinku LYao X(2023)On the validity of retrospective predictive performance evaluation procedures in just-in-time software defect predictionEmpirical Software Engineering10.1007/s10664-023-10341-828:5Online publication date: 18-Sep-2023
https://doi.org/10.1007/s10664-023-10341-8
Falessi DLaureani SÇarka JEsposito MCosta D(2023)Enhancing the defectiveness prediction of methods and classes via JITEmpirical Software Engineering10.1007/s10664-022-10261-z28:2Online publication date: 31-Jan-2023
https://dl.acm.org/doi/10.1007/s10664-022-10261-z
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents