-
The Probabilistic Bounds on the Feasibility of the Defect Prediction Models in Real-World Testing Environments
Authors:
Umamaheswara Sharma B,
Ravichandra Sadam
Abstract:
The research on developing software defect prediction (SDP) models is targeted at reducing the workload on the tester and, thereby, the time spent on the targeted module. However, while a considerable amount of research has been done on developing prediction models or attempting to mitigate the related issues in developing prediction models, it is still unknown whether the developed prediction mod…
▽ More
The research on developing software defect prediction (SDP) models is targeted at reducing the workload on the tester and, thereby, the time spent on the targeted module. However, while a considerable amount of research has been done on developing prediction models or attempting to mitigate the related issues in developing prediction models, it is still unknown whether the developed prediction model really works in real-world testing environments or not. With this article, we bridge this research gap of finding the feasibility of the developed binary defect prediction model in the real-world testing environments. Because machine learning (ML) applications span over many domains, we hope this article may provide sufficient ground to do research on analysing the feasibility of developed prediction models in the related applications in real-time scenarios.
△ Less
Submitted 16 January, 2023;
originally announced January 2023.
-
Do the Defect Prediction Models Really Work?
Authors:
Umamaheswara Sharma B,
Ravichandra Sadam
Abstract:
You may develop a potential prediction model, but how can I trust your model that it will benefit my software?. Using a software defect prediction (SDP) model as a tool, we address this fundamental problem in machine learning research. This is a preliminary work targeted at providing an analysis of the developed binary SDP model in real-time working environments.
You may develop a potential prediction model, but how can I trust your model that it will benefit my software?. Using a software defect prediction (SDP) model as a tool, we address this fundamental problem in machine learning research. This is a preliminary work targeted at providing an analysis of the developed binary SDP model in real-time working environments.
△ Less
Submitted 13 January, 2023;
originally announced January 2023.
-
Towards Developing and Analysing Metric-Based Software Defect Severity Prediction Model
Authors:
Umamaheswara Sharma B,
Ravichandra Sadam
Abstract:
In a critical software system, the testers have to spend an enormous amount of time and effort to maintain the software due to the continuous occurrence of defects. Among such defects, some severe defects may adversely affect the software. To reduce the time and effort of a tester, many machine learning models have been proposed in the literature, which use the documented defect reports to automat…
▽ More
In a critical software system, the testers have to spend an enormous amount of time and effort to maintain the software due to the continuous occurrence of defects. Among such defects, some severe defects may adversely affect the software. To reduce the time and effort of a tester, many machine learning models have been proposed in the literature, which use the documented defect reports to automatically predict the severity of the defective software modules. In contrast to the traditional approaches, in this work we propose a metric-based software defect severity prediction (SDSP) model that uses a self-training semi-supervised learning approach to classify the severity of the defective software modules. The approach is constructed on a mixture of unlabelled and labelled defect severity data. The self-training works on the basis of a decision tree classifier to assign the pseudo-class labels to the unlabelled instances. The predictions are promising since the self-training successfully assigns the suitable class labels to the unlabelled instances.
On the other hand, numerous research studies have covered proposing prediction approaches as well as the methodological aspects of defect severity prediction models, the gap in estimating project attributes from the prediction model remains unresolved. To bridge the gap, we propose five project specific measures such as the Risk-Factor (RF), the Percent of Saved Budget (PSB), the Loss in the Saved Budget (LSB), the Remaining Service Time (RST) and Gratuitous Service Time (GST) to capture project outcomes from the predictions. Similar to the traditional measures, these measures are also calculated from the observed confusion matrix. These measures are used to analyse the impact that the prediction model has on the software project.
△ Less
Submitted 10 October, 2022;
originally announced October 2022.
-
How Far Does the Predictive Decision Impact the Software Project? The Cost, Service Time, and Failure Analysis from a Cross-Project Defect Prediction Model
Authors:
Umamaheswara Sharma B,
Ravichandra Sadam
Abstract:
Context: Cross-project defect prediction (CPDP) models are being developed to optimize the testing resources. Objectives: Proposing an ensemble classification framework for CPDP as many existing models are lacking with better performances and analysing the main objectives of CPDP from the outcomes of the proposed classification framework. Method: For the classification task, we propose a bootstrap…
▽ More
Context: Cross-project defect prediction (CPDP) models are being developed to optimize the testing resources. Objectives: Proposing an ensemble classification framework for CPDP as many existing models are lacking with better performances and analysing the main objectives of CPDP from the outcomes of the proposed classification framework. Method: For the classification task, we propose a bootstrap aggregation based hybrid-inducer ensemble learning (HIEL) technique that uses probabilistic weighted majority voting (PWMV) strategy. To know the impact of HIEL on the software project, we propose three project-specific performance measures such as percent of perfect cleans (PPC), percent of non-perfect cleans (PNPC), and false omission rate (FOR) from the predictions to calculate the amount of saved cost, remaining service time, and percent of the failures in the target project. Results: On many target projects from PROMISE, NASA, and AEEEM repositories, the proposed model outperformed recent works such as TDS, TCA+, HYDRA, TPTL, and CODEP in terms of F-measure. In terms of AUC, the TCA+ and HYDRA models stand as strong competitors to the HIEL model. Conclusion: For better predictions, we recommend ensemble learning approaches for the CPDP models. And, to estimate the benefits from the CPDP models, we recommend the above project-specific performance measures.
△ Less
Submitted 28 September, 2022;
originally announced September 2022.
