Abstract
As shown in the bibliography, training an ensemble of networks is an interesting way to improve the performance. The two key factors to design an ensemble are how to train the individual networks and how to combine the different outputs of the nets. In this paper, we focus on the combination methods. We study the performance of fourteen different combination methods for ensembles of the type “simple ensemble” (SE) and “decorrelated” (DECO). In the case of the “SE” and low number of networks in the ensemble, the method Zimmermann gets the best performance. When the number of networks is in the range of 9 and 20 the weighted average is the best alternative. Finally, in the case of the ensemble “DECO” the best performing method is averaging.
This research was supported by the project MAPACI TIC2002-02273 of CICYT in Spain.
An erratum to this chapter can be found at http://dx.doi.org/10.1007/11550907_163 .
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Tumer, K., Ghosh, J.: Error correlation and error reduction in ensemble classifiers. Connection Science 8(3,4), 385–404 (1996)
Raviv, Y., Intrator, N.: Bootstrapping with Noise: An Effective Regularization Technique. Connection Science 8(3,4), 355–372 (1996)
Fernandez-Redondo, M., Hernández-Espinosa, C., Torres-Sospedra, J.: Classification by Multilayer Feedforward ensembles. In: Yin, F.-L., Wang, J., Guo, C. (eds.) ISNN 2004. LNCS, vol. 3173, pp. 852–857. Springer, Heidelberg (2004)
Verikas, A., Lipnickas, A., Malmqvist, K., Bacauskiene, M., Gelzinis, A.: Soft Combination of neural classifiers: A comparative study. Pattern Recognition Letters 20, 429–444 (1999)
Rosen, B.: Ensemble Learning Using Decorrelated Neural Networks. Connection Science 8(3,4), 373–383 (1996)
Xu, L., Krzyzak, A., SUen, C.Y.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. on Systems, Man and Cybernetics 22(3), 418–435 (1992)
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. Advances in Neural Information Processing Systems 7, 231–238 (1995)
Zymmermann, H.J., Zysno, P.: Decision and evaluations by hierarchical aggregation of information. Fuzzy Sets and Systems 10(3), 243–260 (1984)
Jimenez, D.: Dynamically Weighted Ensemble Neural Network for Classification. IEEE World Congress on Computational Intelligence 1, 753–756 (1998)
Wanas, N.M., Kamel, M.S.: Decision Fusion in Neural Network Ensembles. In: International Joint Conference on Neural Networks, vol. 4, pp. 2952–2957 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Torres-Sospedra, J., Fernández-Redondo, M., Hernández-Espinosa, C. (2005). Combination Methods for Ensembles of MF. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds) Artificial Neural Networks: Formal Models and Their Applications – ICANN 2005. ICANN 2005. Lecture Notes in Computer Science, vol 3697. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11550907_22
Download citation
DOI: https://doi.org/10.1007/11550907_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28755-1
Online ISBN: 978-3-540-28756-8
eBook Packages: Computer ScienceComputer Science (R0)