Automated Parameter Tuning of Artificial Neural Networks for Software Defect Prediction

Defect prediction can help predict defect-prone software modules and improve the efficiency and accuracy of defect location and repair, which plays an extremely important role in software quality assurance. Artificial Neural Networks (ANNs), a family of powerful machine learning regression or classification models, have been widely applied for defect prediction. However, the performance of these models will be degraded if they use suboptimal default parameter settings (e.g., the number of units in the hidden layer). This paper utilizes an automated parameter tuning technique-Caret to optimize parameter settings. In our study, 30 datasets are downloaded from the Tera-PROMISE Repository. According to the characteristics of the datasets, we select key features (metrics) as predictors to train defect prediction models. The experiment applies feed-forward, single hidden layer artificial neural network as classifier to build different defect prediction models respectively with optimized parameter settings and with default parameter settings. Confusion matrix and ROC curve are used for evaluating the quality of the models above. The results show that the models trained with optimized parameter settings outperform the models trained with default parameter settings. Hence, we suggest that researchers should pay attention to tuning parameter settings by Caret for ANNs instead of using suboptimal default settings if they select ANNs for training models in the future defect prediction studies.