Article

Looking for bugs in all the right places

Authors:

Robert M. Bell,

Thomas J. Ostrand,

Elaine J. WeyukerAuthors Info & Claims

ISSTA '06: Proceedings of the 2006 international symposium on Software testing and analysis

Pages 61 - 72

https://doi.org/10.1145/1146238.1146246

Published: 21 July 2006 Publication History

Abstract

We continue investigating the use of a negative binomial regression model to predict which files in a large industrial software system are most likely to contain many faults in the next release. A new empirical study is described whose subject is an automated voice response system. Not only is this system's functionality substantially different from that of the earlier systems we studied (an inventory system and a service provisioning system), it also uses a significantly different software development process. Instead of having regularly scheduled releases as both of the earlier systems did, this system has what are referred to as "continuous releases." We explore the use of three versions of the negative binomial regression model, as well as a simple lines-of-code based model, to make predictions for this system and discuss the differences observed from the earlier studies. Despite the different development process, the best version of the prediction model was able to identify, over the lifetime of the project, 20% of the system's files that contained, on average, nearly three quarters of the faults that were detected in the system's next releases.

References

[1]

E.N. Adams. Optimizing Preventive Service of Software Products. IBM J. Res. Develop., Vol 28, No 1, Jan 1984, pp. 2--14.

Digital Library

[2]

V.R. Basili and B.T. Perricone. Software Errors and Complexity: An Empirical Investigation. Communications of the ACM, Vol 27, No 1, Jan 1984, pp. 42--52.

Digital Library

[3]

G. Denaro and M. Pezze. An Empirical Evaluation of Fault-Proneness Models. Proc. International Conf on Software Engineering (ICSE2002), Miami, USA, May 2002.

[4]

S.G. Eick, T.L. Graves, A.F. Karr, J.S. Marron, A. Mockus. Does Code Decay? Assessing the Evidence from Change Management Data. IEEE Trans. on Software Engineering, Vol 27, No. 1, Jan 2001, pp. 1--12.

Digital Library

[5]

N.E. Fenton and N. Ohlsson. Quantitative Analysis of Faults and Failures in a Complex Software System. IEEE Trans. on Software Engineering, Vol 26, No 8, Aug 2000, pp. 797--814.

Digital Library

[6]

T.L. Graves, A.F. Karr, J.S. Marron, and H. Siy. Predicting Fault Incidence Using Software Change History. IEEE Trans. on Software Engineering, Vol 26, No. 7, July 2000, pp. 653--661.

Digital Library

[7]

L. Guo, Y. Ma, B. Cukic, H. Singh. Robust Prediction of Fault-Proneness by Random Forests. Proc. ISSRE 2004, Saint-Malo, France, Nov. 2004.

Digital Library

[8]

L. Hatton. Reexamining the Fault Density - Component Size Connection. IEEE Software, March/April 1997, pp. 89--97.

Digital Library

[9]

T.M. Khoshgoftaar, E.B. Allen, K.S. Kalaichelvan, N. Goel. Early Quality Prediction: A Case Study in Telecommunications. IEEE Software, Jan 1996, pp. 65--71.

Digital Library

[10]

T.M. Khoshgoftaar, E.B. Allen, J. Deng. Using Regression Trees to Classify Fault-Prone Software Modules. IEEE Trans. on Reliability, Vol 51, No. 4, Dec 2002, pp. 455--462.

[11]

P. McCullagh and J.A. Nelder. Generalized Linear Models, Second Edition, Chapman and Hall, London, 1989.

[12]

A. Mockus and D.M. Weiss. Predicting Risk of Software Changes. Bell Labs Technical Journal, April-June 2000, pp. 169--180.

[13]

K H. Moller and D.J. Paulish. An Empirical Investigation of Software Fault Distribution. Proc. IEEE First International Software Metrics Symposium, Baltimore, Md., May 21-22, 1993, pp. 82--90.

[14]

J.C. Munson and T.M. Khoshgoftaar. The Detection of Fault-Prone Programs. IEEE Trans. on Software Engineering, Vol 18, No 5, May 1992, pp. 423--433.

Digital Library

[15]

N. Ohlsson and H. Alberg. Predicting Fault-Prone Software Modules in Telephone Switches. IEEE Trans. on Software Engineering, Vol 22, No 12, December 1996, pp. 886--894.

Digital Library

[16]

T. Ostrand and E.J. Weyuker. The Distribution of Faults in a Large Industrial Software System. Proc. ACM/International Symposium on Software Testing and Analysis (ISSTA2002), Rome, Italy, July 2002, pp. 55--64.

Digital Library

[17]

T. Ostrand, E.J. Weyuker, and R.M. Bell. Using Static Analysis to Determine Where to Focus Dynamic Testing Effort. Proc. IEE/Workshop Dynamic Analysis (WODA 04), May 2004.

[18]

T.J. Ostrand, E.J. Weyuker, and R.M. Bell. Where the Bugs Are. Proc. ACM/International Symposium on Software Testing and Analysis (ISSTA2004), Boston, MA, July 2004.

Digital Library

[19]

T.J. Ostrand, E.J. Weyuker, and R.M. Bell. Predicting the Location and Number of Faults in Large Software Systems. IEEE Trans. on Software Engineering, Vol 31, No 4, April 2005.

Digital Library

[20]

T.J. Ostrand, E.J. Weyuker, and R.M. Bell. A Different View of Fault Prediction. Proc. 29th IEEE Annual International Computer Software and Applications Conference (COMPSAC 2005), Edinburgh, Scotland, July, 2005.

Digital Library

[21]

M. Pighin and A. Marzona. An Empirical Analysis of Fault Persistence Through Software Releases. Proc. IEEE/ACM ISESE 2003, pp. 206--212.

Digital Library

[22]

SAS Institute Inc. SAS/STAT 9.1 User's Guide, SAS Institute, Cary, NC, 2004.

Cited By

Zhong YShi MHe JFang CChen Z(2023)Security‐based code smell definition, detection, and impact quantification in AndroidSoftware: Practice and Experience10.1002/spe.325753:11(2296-2321)Online publication date: 9-Sep-2023
https://doi.org/10.1002/spe.3257
Fatema KSyeed MHammouda I(2021)Demography of Open Source Software Prediction Models and TechniquesResearch Anthology on Usage and Development of Open Source Software10.4018/978-1-7998-9158-1.ch033(620-652)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-9158-1.ch033
Malhotra RNishant NGurha SRathi V(2021)Application of Particle Swarm Optimization for Software Defect Prediction Using Object Oriented Metrics2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence)10.1109/Confluence51648.2021.9377116(88-93)Online publication date: 28-Jan-2021
https://doi.org/10.1109/Confluence51648.2021.9377116
Show More Cited By

Index Terms

Looking for bugs in all the right places
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Where the bugs are
ISSTA '04: Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis

The ability to predict which files in a large software system are most likely to contain the largest numbers of faults in the next release can be a very valuable asset. To accomplish this, a negative binomial regression model using information from ...
Where the bugs are

The ability to predict which files in a large software system are most likely to contain the largest numbers of faults in the next release can be a very valuable asset. To accomplish this, a negative binomial regression model using information from ...
Automating algorithms for the identification of fault-prone files
ISSTA '07: Proceedings of the 2007 international symposium on Software testing and analysis

This research investigates ways of predicting which files would be most likely to contain large numbers of faults in the next release of a large industrial software system. Previous work involved making predictions using several different models ranging ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA '06: Proceedings of the 2006 international symposium on Software testing and analysis

July 2006

274 pages

ISBN:1595932631

DOI:10.1145/1146238

General Chair:
Lori Pollock
University of Delaware, USA
,
Program Chair:
Mauro Pezzè
Università degli Studi di Milano Bicocca, Italy

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ISSTA06

Sponsor:

ISSTA06: International Symposium on Software Testing and Analysis 2006

July 17 - 20, 2006

Maine, Portland, USA

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

70
Total Citations
View Citations
1,046
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhong YShi MHe JFang CChen Z(2023)Security‐based code smell definition, detection, and impact quantification in AndroidSoftware: Practice and Experience10.1002/spe.325753:11(2296-2321)Online publication date: 9-Sep-2023
https://doi.org/10.1002/spe.3257
Fatema KSyeed MHammouda I(2021)Demography of Open Source Software Prediction Models and TechniquesResearch Anthology on Usage and Development of Open Source Software10.4018/978-1-7998-9158-1.ch033(620-652)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-9158-1.ch033
Malhotra RNishant NGurha SRathi V(2021)Application of Particle Swarm Optimization for Software Defect Prediction Using Object Oriented Metrics2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence)10.1109/Confluence51648.2021.9377116(88-93)Online publication date: 28-Jan-2021
https://doi.org/10.1109/Confluence51648.2021.9377116
Cui CLiu BXiao PWang S(2020)Can Defect Prediction Be Useful for Coarse-Level Tasks of Software Testing?Applied Sciences10.3390/app1015537210:15(5372)Online publication date: 4-Aug-2020
https://doi.org/10.3390/app10155372
Rathore SKumar S(2019)An Approach for the Prediction of Number of Software Faults Based on the Dynamic Selection of Learning TechniquesIEEE Transactions on Reliability10.1109/TR.2018.286420668:1(216-236)Online publication date: Mar-2019
https://doi.org/10.1109/TR.2018.2864206
Rathore SKumar S(2019)A study on software fault prediction techniquesArtificial Intelligence Review10.1007/s10462-017-9563-551:2(255-327)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1007/s10462-017-9563-5
Rathore SKumar SRathore SKumar S(2019)Techniques Used for the Prediction of Number of FaultsFault Prediction Modeling for the Prediction of Number of Software Faults10.1007/978-981-13-7131-8_2(11-29)Online publication date: 4-Apr-2019
https://doi.org/10.1007/978-981-13-7131-8_2
Fatema KSyeed MHammouda I(2018)Demography of Open Source Software Prediction Models and TechniquesOptimizing Contemporary Application and Processes in Open Source Software10.4018/978-1-5225-5314-4.ch002(24-56)Online publication date: 2018
https://doi.org/10.4018/978-1-5225-5314-4.ch002
Ramler RHimmelbauer JNatschläger T(2018)Building Defect Prediction Models in PracticeComputer Systems and Software Engineering10.4018/978-1-5225-3923-0.ch014(324-350)Online publication date: 2018
https://doi.org/10.4018/978-1-5225-3923-0.ch014
Di Nucci DPalomba FDe Rosa GBavota GOliveto RDe Lucia A(2018)A Developer Centered Bug Prediction ModelIEEE Transactions on Software Engineering10.1109/TSE.2017.265974744:1(5-24)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1109/TSE.2017.2659747
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten