Abstract
The paper presents a cost-sensitive classifier ensemble pruning method, which employs a genetic algorithm to choose the most promising ensemble. In this study the pruning algorithm considers constraints put on the cost of selected features, which is the one of the key-problems in the real-life decision support systems, especially dedicated medical support systems. The proposed method takes into consideration both the overall classification accuracy and the cost constraints, returning balanced solution for the problem at hand. Additionally, also to boost the value of the exploitation cost, we propose to use cost-sensitive decision trees as the base classifiers. The pruning algorithm was evaluated on the basis of the comprehensive computer experiments run on cost-sensitive medical benchmark datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: Ensemble diversity measures and their application to thinning. Information Fusion 6(1), 49–62 (2005)
Dai, Q.: A competitive ensemble pruning approach based on cross-validation technique. Knowl.-Based Syst. 37, 394–414 (2013)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001)
Frank, A., Asuncion, A.: UCI machine learning repository (2010). http://archive.ics.uci.edu/ml
Gabrys, B., Ruta, D.: Genetic algorithms in classifier fusion. Appl. Soft Comput. 6(4), 337–347 (2006)
García, S., Fernández, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Inf. Sci. 180(10), 2044–2064 (2010)
Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20, 832–844 (1998)
Jackowski, K., Krawczyk, B., Wozniak, M.: Improved adaptive splitting and selection: The hybrid training method of a classifier based on a feature space partitioning. International Journal of Neural Systems 24(03), 1430007 (2014)
Jackowski, K., Krawczyk, B., Woźniak, M.: Cost-sensitive splitting and selection method for medical decision support system. In: Yin, H., Costa, J.A.F., Barreto, G. (eds.) IDEAL 2012. LNCS, vol. 7435, pp. 850–857. Springer, Heidelberg (2012)
Krawczyk, B., Woźniak, M.: Designing cost-sensitive ensemble – genetic approach. In: Choraś, R.S. (ed.) Image Processing and Communications Challenges 3. AISC, vol. 102, pp. 227–234. Springer, Heidelberg (2011)
Lirov, Y., Yue, O.-C.: Automated network troubleshooting knowledge acquisition. Applied Intelligence 1, 121–132 (1991)
Michalewicz, Z.: Genetic algorithms + data structures = evolution programs, 3rd edn. Springer, London (1996)
Núñez, M.: The use of background knowledge in decision tree induction. Mach. Learn. 6(3), 231–250 (1991)
Núñez, M.: Economic induction: A case study. In: EWSL, pp. 139–145 (1988)
Penar, W., Wozniak, M.: Cost-sensitive methods of constructing hierarchical classifiers. Expert Systems 27(3), 146–155 (2010)
Peng, Y., Huang, Q., Jiang, P., Jiang, J.: Cost-sensitive ensemble of support vector machines for effective detection of microcalcification in breast cancer diagnosis. In: Wang, L., Jin, Y. (eds.) FSKD 2005. LNCS (LNAI), vol. 3614, pp. 483–493. Springer, Heidelberg (2005)
Tan, M., Schlimmer, J.C.: Cost-sensitive concept learning of sensor use in approach and recognition. In: Proceedings of the Sixth International Workshop on Machine Learning, pp. 392–395. Morgan Kaufmann Publishers Inc., San Francisco (1989)
Turney, P.D.: Cost-sensitive classification: empirical evaluation of a hybrid genetic decision tree induction algorithm. J. Artif. Int. Res. 2(1), 369–409 (1995)
Verdenius, F.: A method for inductive cost optimization. In: Proceedings of the European Working Session on Learning on Machine Learning, EWSL 1991, pp. 179–191. Springer-Verlag New York Inc., New York (1991)
Acknowledgments
This work was supported by the Polish National Science Center under the grant no. DEC-2013/09/B/ST6/02264.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Krawczyk, B., Woźniak, M. (2015). Pruning Ensembles with Cost Constraints. In: Nguyen, N., Trawiński, B., Kosala, R. (eds) Intelligent Information and Database Systems. ACIIDS 2015. Lecture Notes in Computer Science(), vol 9011. Springer, Cham. https://doi.org/10.1007/978-3-319-15702-3_49
Download citation
DOI: https://doi.org/10.1007/978-3-319-15702-3_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15701-6
Online ISBN: 978-3-319-15702-3
eBook Packages: Computer ScienceComputer Science (R0)