Abstract
Testing involves examining the behavior of a system in order to discover potential faults. Given an input for a system, the challenge of distinguishing the correct behavior from potentially incorrect one, is called the “test oracle problem”. Metamorphic testing has shown great potential in overcoming the test oracle problem. In this work, we apply metamorphic testing to validate experimentally machine learning classification algorithms, namely Naïve Bayes (NB) and k-Nearest Neighbor (k-NN) individually and in combination (i.e., ensemble classifications methods), using real-world biomedical datasets. Furthermore, advanced feature selection techniques and synthetic minority over-sampling technique (SMOTE) are used in order to generate our test suite and meet the requirements of the specified metamorphic relations. While, this study reveal that NB and k-NN satisfy the specified metamorphic relations, it also concludes that it is not compulsory that the metamorphic relations that are necessary for NB and k-NN individually, are also necessary for their ensemble classifier.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16(1), 321–357 (2002)
Chen, T.Y., Cheung, S.C., Yiu, S.: Metamorphic testing: a new approach for generating next test cases. Technical Report HKUST-CS98-01, Department of Computer Science, Hong Kong University of Science and Technology (1998)
Davis, M.D., Weyuker, E.J.: Pseudo-oracles for non-testable programs. In: Proceedings of the ACM 1981 Conference, NY, USA, pp. 254–257 (1981). http://doi.acm.org/10.1145/800175.809889
Hall, M.A.: Correlation-based feature selection for machine learning. Ph.D. thesis, The University of Waikato (1999)
He, H., Garcia, E., et al.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)
Lichman, M.: UCI machine learning repository. http://archive.ics.uci.edu/ml (2013). Accessed May 2017
Murphy, C., Kaiser, G.E., Hu, L., Wu, L.: Properties of machine learning applications for use in metamorphic testing. In: Proceedings of the Twentieth International Conference on Software Engineering & Knowledge Engineering (SEKE 2008), San Francisco, CA, USA, 1–3 July 2008, pp. 867–872 (2008)
Ramana, B.V., Babu, M.S.P., Venkateswarlu, N.: A critical study of selected classification algorithms for liver disease diagnosis. Int. J. Database Manag. Syst. 3(2), 101–114 (2011)
Segura, S., Fraser, G., Sanchez, A.B., Ruiz-Cortés, A.: A survey on metamorphic testing. IEEE Trans. Softw. Eng. 42(9), 805–824 (2016)
Sesmero, M.P., Ledezma, A.I., Sanchis, A.: Generating ensembles of heterogeneous classifiers using stacked generalization. Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 5(1), 21–34 (2015)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Burlington (2005)
Xie, X., Ho, J., Murphy, C., Kaiser, G., Xu, B., Chen, T.Y.: Application of metamorphic testing to supervised classifiers. In: 9th International Conference on Quality Software QSIC 2009, pp. 135–144. IEEE (2009)
Xie, X., Ho, J.W., Murphy, C., Kaiser, G., Xu, B., Chen, T.Y.: Testing and validating machine learning classifiers by metamorphic testing. J. Syst. Softw. 84(4), 544–558 (2011)
Zhou, Z.Q., Huang, D., Tse, T., Yang, Z., Huang, H., Chen, T.: Metamorphic testing and its applications. In: Proceedings of the 8th International Symposium on Future Software Technology (ISFST 2004), pp. 346–351 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Al-Azani, S., Hassine, J. (2017). Validation of Machine Learning Classifiers Using Metamorphic Testing and Feature Selection Techniques. In: Phon-Amnuaisuk, S., Ang, SP., Lee, SY. (eds) Multi-disciplinary Trends in Artificial Intelligence. MIWAI 2017. Lecture Notes in Computer Science(), vol 10607. Springer, Cham. https://doi.org/10.1007/978-3-319-69456-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-69456-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69455-9
Online ISBN: 978-3-319-69456-6
eBook Packages: Computer ScienceComputer Science (R0)