research-article

How do fixes become bugs?

Authors:

Shankar Pasupathy,

Lakshmi BairavasundaramAuthors Info & Claims

ESEC/FSE '11: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering

Pages 26 - 36

https://doi.org/10.1145/2025113.2025121

Published: 09 September 2011 Publication History

Abstract

Software bugs affect system reliability. When a bug is exposed in the field, developers need to fix them. Unfortunately, the bug-fixing process can also introduce errors, which leads to buggy patches that further aggravate the damage to end users and erode software vendors' reputation.

This paper presents a comprehensive characteristic study on incorrect bug-fixes from large operating system code bases including Linux, OpenSolaris, FreeBSD and also a mature commercial OS developed and evolved over the last 12 years, investigating not only themistake patterns during bug-fixing but also the possible human reasons in the development process when these incorrect bug-fixes were introduced. Our major findings include: (1) at least 14.8%--24.4% of sampled fixes for post-release bugs in these large OSes are incorrect and have made impacts to end users. (2) Among several common bug types, concurrency bugs are the most difficult to fix correctly: 39% of concurrency bug fixes are incorrect. (3) Developers and reviewers for incorrect fixes usually do not have enough knowledge about the involved code. For example, 27% of the incorrect fixes are made by developers who have never touched the source code files associated with the fix. Our results provide useful guidelines to design new tools and also to improve the development process. Based on our findings, the commercial software vendor whose OS code we evaluated is building a tool to improve the bug fixing and code reviewing process.

References

[1]

Microsoft security bulletin. http://www.microsoft.com/technet/security/current.aspx.

[2]

After buggy patch, criminals exploit Windows flaw. http://www.infoworld.com/d/security-central/after-buggy-patch-criminals-exploit-windows-flaw-848.

[3]

J. Anvik, L. Hiew, and G. C. Murphy. Who should fix this bug? In ICSE'06.

Digital Library

[4]

Apple updates leopard again. http://voices.washingtonpost.com/fasterforward/2008/~02/apple_updates_leopardagain.html.

[5]

A. Bachmann, C. Bird, F. Rahman, P. Devanbu, and A. Bernstein. The missing links: Bugs and bug-fix commits. In FSE'10.

Digital Library

[6]

M. J. Baker and S. G. Eick. Visualizing software systems. In ICSE'94.

Digital Library

[7]

C. Bird, N. Nagappan, P. Devanbu, H. Gall, and B. Murphy. Does distributed development affect software quality? An empirical case study of Windows Vista. In ICSE'09.

Digital Library

[8]

Buggy McAfee update whacks Windows XP PCs. http://news.cnet.com/8301-1009_3-20003074-83.html.

[9]

D. Engler and K. Ashcraft. RacerX: effective, static detection of race conditions and deadlocks. In SOSP'03.

Digital Library

[10]

D. Engler, D. Y. Chen, S. Hallem, A. Chou, and B. Chelf. Bugs as deviant behavior: a general approach to inferring errors in systems code. In SOSP'01.

Digital Library

[11]

M. Fischer, M. Pinzger, and H. Gall. Populating a release history database from version control and bug tracking systems. In ICSM'03.

Digital Library

[12]

K. B. Gallagher and J. R. Lyle. Using program slicing in software maintenance. IEEE Transactions on Software Engineering, 17(8):751--761, 1991.

Digital Library

[13]

Z. Gu, E. T.Barr, D. J.Hamilton, and Z. Su. Has the bug really been fixed? In ICSE'10.

Digital Library

[14]

M. Harman, D. Binkley, and S. Danicic. Amorphous program slicing. Journal of Systems and Software, 68(1):45--64, October 2003.

Digital Library

[15]

Intel reissues buggy patch. http://www.pcworld.com/businesscenter/article/126918/rss.html.

[16]

D. Kawrykow and M. P. Robillard. Non-essential changes in version histories. In ICSE'11, May 2011.

Digital Library

[17]

S. Kim, K. Pan, and J. E. James Whitehead. Memories of bug fixes. In FSE'06, November 2006.

Digital Library

[18]

S. Kim, E. J. Whitehead, Jr., and Y. Zhang. Classifying Software Changes: Clean or Buggy? IEEE Trans. Software Engineering, 34(2):181--196, March 2008.

Digital Library

[19]

Z. Li, L. Tan, X. Wang, S. Lu, Y. Zhou, and C. Zhai. Have things changed now? An empirical study of bug characteristics in modern open source software. In ASID'06.

Digital Library

[20]

Z. Li and Y. Zhou. PR-Miner: Automatically extracting implicit programming rules and detecting violations in large software code. In FSE'05.

Digital Library

[21]

S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from mistakes -- a comprehensive study on real world concurrency bug characteristics. In ASPLOS, March 2008.

Digital Library

[22]

McAfee to reimburse customers for bad patch. http://www.computerworlduk.com/technology/security-products/~prevention/news/index.cfm?newsId=20005.

[23]

S. McCamant and M. D. Ernst. Predicting problems caused by component upgrades. In FSE'03.

Digital Library

[24]

D. W. McDonald and M. S. Ackerman. Expertise recommender: a flexible recommendation system and architecture. In CSCW'00.

Digital Library

[25]

A. Meneely and L. Williams. Secure open source collaboration: An empirical study of linus's law. In CCS'09.

Digital Library

[26]

A. Mockus and D. M. Weiss. Predicting risk of software changes. Bell Labs Technical Journal, 5:169--180, 2000.

[27]

N. Nagappan, B. Murphy, and V. R. Basili. The influence of organizational structure on software quality. In ICSE'08.

Digital Library

[28]

T. T. Nguyen, H. A. Nguyen, N. H. Pham, J. Al-Kofahi, and T. N. Nguyen. Recurring bug fixes in object-oriented programs. In ICSE'10, May 2010.

Digital Library

[29]

A. Orso, N. Shi, and M. J. Harrold. Scaling regression testing to large software systems. In FSE'04.

Digital Library

[30]

Y. Padioleau, J. Lawall, R. R. Hansen, and G. Muller. Documenting and automating collateral evolutions in linux device drivers. In Eurosys'08.

Digital Library

[31]

K. Pan, S. Kim, and J. E. James Whitehead. Toward an understanding of bug fix patterns. Empirical Software Engineering, 14(3):286--315, November 2009.

Digital Library

[32]

J. H. Perkins, S. Kim, S. Larseng, S. Amarasinghe, J. Bachrach, M. Carbin, C. Pachecod, F. Sherwood, S. Sidiroglou, G. Sullivan, W.-F. Wong, Y. Zibin, M. D. Ernst, and M. Rinard. Automatically patching errors in deployed software. In SOSP'09, October 2009.

Digital Library

[33]

R. Purushothaman and D. E. Perry. Towards understanding the rhetoric of small changes. In MSR'04.

[34]

F. Rahman and P. Devanbu. Ownership and experience in fix-inducing code. In UC Davis Department of Computer Science, Technical Report CSE-2010-4, 2010.

[35]

G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. Prioritizing test cases for regression testing. IEEE Transactions on Software Engineering, 27:929--948, 2001.

Digital Library

[36]

J. Sliwerski, T. Zimmermann, and A. Zeller. When do changes induce fixes? In MSR'05.

Digital Library

[37]

J. Sliwerski, T. Zimmermann, and A. Zeller. Hatari: Raising risk awareness (research demonstration). In FSE'05, September 2005.

Digital Library

[38]

L. Tan, D. Yuan, and Y. Zhou. /* icomment: Bugs or bad comments? */. In SOSP, October 2007.

Digital Library

[39]

J. Tucek, W. Xiong, and Y. Zhou. Efficient online validationwith delta execution. In ASPLOS'09.

Digital Library

[40]

D. Cubrani? and G. C. Murphy. Hipikat: Recommending pertinent software development artifacts. In ICSE'03.

Digital Library

[41]

M. Weiser. Program slicing. In ICSE'83.

[42]

Z. Yin, M. Caesar, and Y. Zhou. Towards understanding bugs in open source router software. ACM SIGCOMM Computer Communication Review, 40(3):34--40, July 2010.

Digital Library

[43]

D. Yuan, H. Mai, W. Xiong, L. Tan, Y. Zhou, and S. Pasupathy. Sherlog: error diagnosis by connecting clues from run-time logs. In ASPLOS'10.

Digital Library

[44]

A. Zeller. Yesterday, my program worked. today, it does not. why? In FSE'99.

Digital Library

Cited By

Jiang MJiang JWu TMa ZLuo XZhou Y(2024)Understanding Vulnerability Inducing Commits of the Linux KernelACM Transactions on Software Engineering and Methodology10.1145/367245233:7(1-28)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672452
Beyer DGrunske LKettl MLingsch-Rosenfeld MRaselimo MSpinellis DConstantinou EBacchelli A(2024)P3: A Dataset of Partial Program FixesProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644889(123-127)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643991.3644889
Abdi JPosluns GZhang GWang BJeffrey MAgrawal KPetrank E(2024)When Is Parallelism Fearless and Zero-Cost with Rust?Proceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3659966(27-40)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3659966
Show More Cited By

Index Terms

How do fixes become bugs?
1. Software and its engineering

Recommendations

Utilizing a multi-developer network-based developer recommendation algorithm to fix bugs effectively
SAC '14: Proceedings of the 29th Annual ACM Symposium on Applied Computing

Recently, bug fixing has become an important part of software maintenance. In large-scale projects, developers rely on bug reports to guide any bug-fixing activities. Due to a great number of bug reports submitted into the bug repository, the workload ...
Do bugs lead to unnaturalness of source code?
ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Texts in natural languages are highly repetitive and predictable because of the naturalness of natural languages. Recent research validated that source code in programming languages is also repetitive and predictable, and naturalness is an inherent ...
Supplementary Bug Fixes vs. Re-opened Bugs
SCAM '14: Proceedings of the 2014 IEEE 14th International Working Conference on Source Code Analysis and Manipulation

A typical bug fixing cycle involves the reporting of a bug, the triaging of the report, the production and verification of a fix, and the closing of the bug. However, previous work has studied two phenomena where more than one fix are associated with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE '11: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering

September 2011

548 pages

ISBN:9781450304436

DOI:10.1145/2025113

General Chair:
Tibor Gyimóthy
University of Szeged, Hungary
,
Program Chair:
Andreas Zeller
Saarland University, Germany

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE'11

Sponsor:

SIGSOFT

ESEC/FSE'11: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering

September 5 - 9, 2011

Szeged, Hungary

Acceptance Rates

Overall Acceptance Rate 17 of 128 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

200
Total Citations
View Citations
1,513
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)14

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang MJiang JWu TMa ZLuo XZhou Y(2024)Understanding Vulnerability Inducing Commits of the Linux KernelACM Transactions on Software Engineering and Methodology10.1145/367245233:7(1-28)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3672452
Beyer DGrunske LKettl MLingsch-Rosenfeld MRaselimo MSpinellis DConstantinou EBacchelli A(2024)P3: A Dataset of Partial Program FixesProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644889(123-127)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643991.3644889
Abdi JPosluns GZhang GWang BJeffrey MAgrawal KPetrank E(2024)When Is Parallelism Fearless and Zero-Cost with Rust?Proceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3659966(27-40)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3659966
Eladawy HLe Goues CBrun YRoychoudhury APaiva AAbreu RStorey M(2024)Automated Program Repair, What Is It Good For? Not Absolutely Nothing!Proceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639095(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639095
Watanabe HKondo MChoi EMizuno O(2024)Benefits and Pitfalls of Token-Level SZZ: An Empirical Study on OSS Projects2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00084(776-786)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00084
Edward ENyamawe AElisa N(2024)On the Impact of Refactorings on Software Attack SurfaceIEEE Access10.1109/ACCESS.2024.340405812(128570-128584)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3404058
Goyal ASardana N(2024)An Integrated Approach using Developer Profiles with Temporal Dynamics for Assignee Recommendation in Non-Reproducible BugsProcedia Computer Science10.1016/j.procs.2024.04.268235(2833-2842)Online publication date: 2024
https://doi.org/10.1016/j.procs.2024.04.268
Maes-Bermejo MSerebrenik AGallego MGortázar FRobles GGonzález Barahona J(2024)Hunting bugs: Towards an automated approach to identifying which change caused a bug through regression testingEmpirical Software Engineering10.1007/s10664-024-10479-z29:3Online publication date: 4-May-2024
https://doi.org/10.1007/s10664-024-10479-z
Wu YLin ZChen YLe DMu DXing XCalandrino JTroncoso C(2023)Mitigating security risks in Linux with KLAUSProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620475(4247-4264)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620475
Franken GVan Goethem TDesmet LJoosen WCalandrino JTroncoso C(2023)A bug's lifeProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620443(3673-3690)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620443
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents