Abstract
Importance of soft computing methods has continuously grown for many years. Particularly machine learning methods have been paid considerable attention in the business sphere and subsequently within the general public in the last decade. Machine learning and its implementation is the object of interest of many commercial subjects, whether they are small companies or large corporations. Consequently, well-educated experts in the area of machine learning are highly sought after on the job market. Most of the technical universities around the world have incorporated the machine learning into their curricula. However, machine learning is a dynamically evolving area and the curricula should be continuously updated. This paper is intended to support this process. Namely, an imbalance data issue, in context of performance measures for binary classification, is opened, and a teaching method covering this problem is presented. The method has been primary designed for undergraduate and graduate students of technical fields; however, it can be easily adopted in curricula of other fields of study, e.g. medicine, economics, or social sciences.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
ROC = receiver operating characteristic.
References
Barandela, R., Sánchez, J.S., García, V., Rangel, E.: Strategies for learning in class imbalance problems. Pattern Recogn. 36(3), 849–851 (2003)
Brodersen, K., Ong, C.S., Stephan, K., Buhmann, J.: The balanced accuracy and its posterior distribution. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 3121–3124, August 2010
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16(1), 321–357 (2002)
Choi, S.S., Cha, S.H., Tappert, C.: A survey of binary similarity and distance measures. J. Systemics Cybern. Inform. 8(1), 43–48 (2010)
Daskalaki, S., Kopanas, I., Avouris, N.M.: Evaluation of classifiers for an uneven class distribution problem. Appl. Artif. Intell. 20(5), 381–417 (2006)
Fawcett, T.: An introduction to ROC analysis. Pattern Recogn. Lett. 27(8), 861–874 (2006)
Flach, P.: Machine Learning: The Art and Science of Algorithms that Make Sense of Data. Cambridge University Press, New York (2012)
Forman, G., Scholz, M.: Apples-to-apples in cross-validation studies: pitfalls in classier performance measurement. SIGKDD Explor. 12(1), 49–57 (2010)
Garcia, V., Mollineda, R.A., Sanchez, J.S.: Theoretical analysis of a performance measure for imbalanced data. In: 2010 20th International Conference on Pattern Recognition, pp. 617–620. IEEE, August 2010
García, V., Sánchez, J.S., Mollineda, R.A., Alejo, R., Sotoca, J.M.: The class imbalance problem in pattern classification and learning. In: Ferrer-Troyano, F.J., et al. (eds.) II Congreso Español de Informática, pp. 283–291. Thomson, Zaragoza (2007)
Goutte, C., Gaussier, E.: A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In: Losada, D.E., Fernández-Luna, J.M. (eds.) ECIR 2005. LNCS, vol. 3408, pp. 345–359. Springer, Heidelberg (2005). doi:10.1007/978-3-540-31865-1_25
Hand, D.J.: Assessing the performance of classification methods. Int. Stat. Rev. 80(3), 400–414 (2012)
Hossin, M., Sulaiman, M.: A review on evaluation metrics for data classification evaluations. Int. J. Data Min. Knowl. Manage. Process 5(2), 1–11 (2015)
Japkowicz, N.: Why question machine learning evaluation methods. In: AAAI Workshop on Evaluation Methods for Machine Learning, pp. 6–11 (2006)
Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6(5), 429–449 (2002)
Kotsiantis, S., Pintelas, P.: Mixture of expert agents for handling imbalanced data sets. Ann. Math. Comput. Teleinformatics 1(1), 46–55 (2003)
Kubat, M.: An Introduction to Machine Learning. Springer, Cham (2015)
Lemnaru, C., Potolea, R.: Imbalanced classification problems: systematic study, issues and best practices. In: Zhang, R., Zhang, J., Zhang, Z., Filipe, J., Cordeiro, J. (eds.) ICEIS 2011. LNBIP, vol. 102, pp. 35–50. Springer, Heidelberg (2012). doi:10.1007/978-3-642-29958-2_3
Mani, I., Zhang, I.: kNN approach to unbalanced data distributions: a case study involving information extraction. In: Proceedings of Workshop on Learning from Imbalanced Datasets (2003)
Powers, D.M.W.: Evaluation: From precision, recall and f-measure to ROC, informedness, markedness & correlation. J. Mach. Learn. Technol. 2(1), 37–63 (2011)
Salzberg, S.L.: On comparing classifiers: a critique of current research and methods. Data Min. Knowl. Disc. 1, 1–12 (1999)
Salzberg, S.L.: On comparing classifiers: pitfalls to avoid and a recommended approach. Data Min. Knowl. Discov. 1(3), 317–328 (1997)
Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inf. Process. Manage. 45(4), 427–437 (2009)
Velez, D.R., White, B.C., Motsinger, A.A., Bush, W.S., Ritchie, M.D., Williams, S.M., Moore, J.H.: A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reduction. Genet. Epidemiol. 31(4), 306–315 (2007)
Acknowledgement
The work has been supported by the Funds of University of Pardubice, Czech Republic, grant No. SGS-2016-017. This support is very gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Škrabánek, P., Majerík, F. (2017). Reflecting on Imbalance Data Issue When Teaching Performance Measures. In: Silhavy, R., Senkerik, R., Kominkova Oplatkova, Z., Prokopova, Z., Silhavy, P. (eds) Artificial Intelligence Trends in Intelligent Systems. CSOC 2017. Advances in Intelligent Systems and Computing, vol 573. Springer, Cham. https://doi.org/10.1007/978-3-319-57261-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-57261-1_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-57260-4
Online ISBN: 978-3-319-57261-1
eBook Packages: EngineeringEngineering (R0)