research-article

The Google FindBugs fixit

Authors:

Nathaniel Ayewah,

William PughAuthors Info & Claims

ISSTA '10: Proceedings of the 19th international symposium on Software testing and analysis

Pages 241 - 252

https://doi.org/10.1145/1831708.1831738

Published: 12 July 2010 Publication History

Abstract

In May 2009, Google conducted a company wide FindBugs "fixit". Hundreds of engineers reviewed thousands of FindBugs warnings, and fixed or filed reports against many of them. In this paper, we discuss the lessons learned from this exercise, and analyze the resulting dataset, which contains data about how warnings in each bug pattern were classified. Significantly, we observed that even though most issues were flagged for fixing, few appeared to be causing any serious problems in production. This suggests that most interesting software quality problems were eventually found and fixed without FindBugs, but FindBugs could have found these problems early, when they are cheap to remediate. We compared this observation to bug trends observed in code snapshots from student projects.

The full dataset from the Google fixit, with confidential details encrypted, will be published along with this paper.

References

[1]

N. Ayewah, D. Hovemeyer, J. D. Morgenthaler, J. Penix, and W. Pugh. Using static analysis to find bugs. IEEE Softw., 25(5):22--29, 2008.

Digital Library

[2]

N. Ayewah and W. Pugh. A report on a survey and study of static analysis users. In DEFECTS '08: Proceedings of the 2008 workshop on Defects in large software systems, pages 1--5, New York, NY, USA, 2008. ACM.

Digital Library

[3]

N. Ayewah and W. Pugh. Using checklists to review static analysis warnings. In DEFECTS '09: Proceedings of the 2nd International Workshop on Defects in Large Software Systems, pages 11--15, New York, NY, USA, 2009. ACM.

Digital Library

[4]

A. Bessey, K. Block, B. Chelf, A. Chou, B. Fulton, S. Hallem, C. Henri-Gros, A. Kamsky, S. McPeak, and D. Engler. A few billion lines of code later: using static analysis to find bugs in the real world. Commun. ACM, 53(2):66--75, 2010.

Digital Library

[5]

S. Boslaugh and D. P. A. Watters. Statistics in a nutshell. O'Reilly & Associates, Inc., Sebastopol, CA, USA, 2008.

Digital Library

[6]

R. P. Buse and W. R. Weimer. A metric for software readability. In ISSTA '08: Proceedings of the 2008 international symposium on Software testing and analysis, pages 121--130, New York, NY, USA, 2008. ACM.

Digital Library

[7]

D. Hovemeyer and W. Pugh. Finding bugs is easy. In OOPSLA '04: Companion to the 19th annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 132--136, New York, NY, USA, 2004. ACM.

Digital Library

[8]

C. Jaspan, I.-C. Chen, and A. Sharma. Understanding the value of program analysis tools. In Companion to the 22nd ACM SIGPLAN conference on Object oriented programming systems and applications companion, pages 963--970, Montreal, Quebec, Canada, 2007. ACM.

Digital Library

[9]

S. Kim and M. D. Ernst. Prioritizing warning categories by analyzing software history. In MSR '07: Proceedings of the Fourth International Workshop on Mining Software Repositories, page 27, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[10]

S. Kim and M. D. Ernst. Which warnings should i fix first? In ESEC-FSE '07: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, pages 45--54, New York, NY, USA, 2007. ACM.

Digital Library

[11]

B. Mediratta and J. Bick. The google way: Give engineers room. The New York Times, Oct 2007.

[12]

N. Nagappan and T. Ball. Static analysis tools as early indicators of pre-release defect density. In ICSE '05: Proceedings of the 27th international conference on Software engineering, pages 580--586, New York, NY, USA, 2005. ACM.

Digital Library

[13]

J. R. Ruthruff, J. Penix, J. D. Morgenthaler, S. Elbaum, and G. Rothermel. Predicting accurate and actionable static analysis warnings: an experimental approach. In ICSE '08: Proceedings of the 30th international conference on Software engineering, pages 341--350, New York, NY, USA, 2008. ACM.

Digital Library

[14]

J. Spacco, D. Hovemeyer, and W. Pugh. Tracking defect warnings across versions. In MSR '06: Proceedings of the 2006 international workshop on Mining software repositories, pages 133--136, New York, NY, USA, 2006. ACM.

Digital Library

[15]

J. Spacco, D. Hovemeyer, W. Pugh, F. Emad, J. K. Hollingsworth, and N. Padua-Perez. Experiences with marmoset: designing and using an advanced submission and testing system for programming courses. In ITICSE '06: Proceedings of the 11th annual SIGCSE conference on Innovation and technology in computer science education, pages 13--17, New York, NY, USA, 2006. ACM.

Digital Library

Cited By

Tan YTian J(2024)A Method for Processing Static Analysis Alarms Based on Deep LearningApplied Sciences10.3390/app1413554214:13(5542)Online publication date: 26-Jun-2024
https://doi.org/10.3390/app14135542
Bhutani VToosi FBuckley J(2024)Analysing the AnalysersApplied Computer Systems10.2478/acss-2024-001329:1(98-111)Online publication date: 15-Aug-2024
https://dl.acm.org/doi/10.2478/acss-2024-0013
Wadhams ZIzurieta CReinhold AFilkov VRay BZhou M(2024)Barriers to Using Static Application Security Testing (SAST) Tools: A Literature ReviewProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering Workshops10.1145/3691621.3694947(161-166)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691621.3694947
Show More Cited By

Index Terms

The Google FindBugs fixit

Recommendations

Using FindBugs on production software
OOPSLA '07: Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion

This poster will present our experiences using FindBugs in production software development environments, including both open source efforts and Google's internal code base. We summarize the defects found, describe the issue of real but trivial defects, ...
Finding bugs in eclipse
OOPSLA '07: Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion

This will be a live demonstration of FindBugs, a static analysis bug finding tool, on the current development version of Eclipse 3.4. FindBugs reports issues such as null pointer dereferences, comparing incompatible types with equals, invalid method ...
Using checklists to review static analysis warnings
DEFECTS '09: Proceedings of the 2nd International Workshop on Defects in Large Software Systems: Held in conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2009)

Static analysis tools find silly mistakes, confusing code, bad practices and property violations. But software developers and organizations may or may not care about all these warnings, depending on how they impact code behavior and other factors. In ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA '10: Proceedings of the 19th international symposium on Software testing and analysis

July 2010

294 pages

ISBN:9781605588230

DOI:10.1145/1831708

General Chair:
Paolo Tonella
Fondazione Bruno Kessler -- IRST, Italy
,
Program Chair:
Alessandro Orso
Georgia Institute of Technology, USA

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '10

Sponsor:

SIGSOFT

ISSTA '10: International Symposium on Software Testing and Analysis

July 12 - 16, 2010

Trento, Italy

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

96
Total Citations
View Citations
2,865
Total Downloads

Downloads (Last 12 months)637
Downloads (Last 6 weeks)57

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tan YTian J(2024)A Method for Processing Static Analysis Alarms Based on Deep LearningApplied Sciences10.3390/app1413554214:13(5542)Online publication date: 26-Jun-2024
https://doi.org/10.3390/app14135542
Bhutani VToosi FBuckley J(2024)Analysing the AnalysersApplied Computer Systems10.2478/acss-2024-001329:1(98-111)Online publication date: 15-Aug-2024
https://dl.acm.org/doi/10.2478/acss-2024-0013
Wadhams ZIzurieta CReinhold AFilkov VRay BZhou M(2024)Barriers to Using Static Application Security Testing (SAST) Tools: A Literature ReviewProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering Workshops10.1145/3691621.3694947(161-166)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691621.3694947
Zhang HPei YLiang STan S(2024)Understanding and Detecting Annotation-Induced Faults of Static AnalyzersProceedings of the ACM on Software Engineering10.1145/36437591:FSE(722-744)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643759
El Karchi JChen HTehraniJamsaz AJannesari APopov MSaillard E(2024)MPI Errors Detection using GNN Embedding and Vector Embedding over LLVM IR2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)10.1109/IPDPS57955.2024.00059(595-607)Online publication date: 27-May-2024
https://doi.org/10.1109/IPDPS57955.2024.00059
Yu XCogo FMcIntosh SGodfrey M(2024)Studying the impact of risk assessment analytics on risk awareness and code review performanceEmpirical Software Engineering10.1007/s10664-024-10443-x29:2Online publication date: 17-Feb-2024
https://dl.acm.org/doi/10.1007/s10664-024-10443-x
Zhang HPei YChen JTan SChandra SBlincoe KTonella P(2023)Statfier: Automated Testing of Static Analyzers via Semantic-Preserving Program TransformationsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616272(237-249)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616272
Mehrpour SLatoza T(2023)A Survey of Tool Support for Working with Design Decisions in CodeACM Computing Surveys10.1145/360786856:2(1-37)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3607868
Guo ZTan TLiu SLiu XLai WYang YLi YChen LDong WZhou Y(2023)Mitigating False Positive Static Analysis Warnings: Progress, Challenges, and OpportunitiesIEEE Transactions on Software Engineering10.1109/TSE.2023.332966749:12(5154-5188)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TSE.2023.3329667
Yedida RKang HTu HYang XLo DMenzies T(2023)How to Find Actionable Static Analysis Warnings: A Case Study With FindBugsIEEE Transactions on Software Engineering10.1109/TSE.2023.323420649:4(2856-2872)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TSE.2023.3234206
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents