Abstract
Multiple classifier systems based on neural networks can give improved generalisation performance as compared with single classifier systems. We examine collaboration in multi-net systems through in-situ learning, exploring how generalisation can be improved through the simultaneous learning in networks and their combination. We present two in-situ trained systems; first, one based upon the simple ensemble, combining supervised networks in parallel, and second, a combination of unsupervised and supervised networks in sequence. Results for these are compared with existing approaches, demonstrating that in-situ trained systems perform better than similar pre-trained systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Blake, C.L., Merz, C.J.U.: Repository of Machine Learning Databases. Irvine, CA.: University of California, Irvine, Department of Information and Computer Sciences (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Bottou, L., Gallinari, P.: A Framework for the Cooperation of Learning Algorithms. In: Lippmann, R.P., Moody, J.E., Touretzky, D.S. (eds.) Advances in Neural Information Processing Systems, vol. 3, pp. 781–788 (1991)
Casey, M.C.: Integrated Learning in Multi-net Systems. Unpublished doctoral thesis. University of Surrey, Guildford (2004)
Drucker, H.: Boosting Using Neural Networks. In: Sharkey, A.J.C. (ed.) Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, pp. 51–78. Springer, London (1999)
Jacobs, R.A., Tanner, M.: Mixtures of X. In: Sharkey, A.J.C. (ed.) Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, pp. 267–295. Springer, Heidelberg (1999)
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On Combining Classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998)
Kohonen, T.: Self-Organized Formation of Topologically Correct Feature Maps. Biological Cybernetics 43, 59–69 (1982)
Liu, Y., Yao, X.: Ensemble Learning via Negative Correlation. Neural Networks 12(10), 1399–1404 (1999)
Liu, Y., Yao, X., Zhao, Q., Higuchi, T.: An Experimental Comparison of Neural Network Ensemble Learning Methods on Decision Boundaries. In: Proceedings of the 2002 International Joint Conference on Neural Networks (IJCNN 2002), vol. 1, pp. 221–226. IEEE Computer Society Press, Los Alamitos (2002)
Partridge, D., Griffith, N.: Multiple Classifier Systems: Software Engineered, Automatically Modular Leading to a Taxonomic Overview. Pattern Analysis and Applications 5(2), 180–188 (2002)
Prechelt, L.: Early Stopping - But When? In: Orr, G.B., Müller, K.-R. (eds.) NIPS-WS 1996. LNCS, vol. 1524, pp. 55–69. Springer, Heidelberg (1998)
Sharkey, A.J.C.: Multi-Net Systems. In: Sharkey, A.J.C. (ed.) Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, pp. 1–30. Springer, London (1999)
Thrun, S.B., Bala, J., Bloedorn, E., Bratko, I., Cestnik, B., Cheng, J., De Jong, K., Dzeroski, S., Fahlman, S.E., Fisher, D., Hamann, R., Kaufman, K., Keller, S., Kononenko, I., Kreuziger, J., Michalski, R.S., Mitchell, T., Pachowicz, P., Reich, Y., Vafaie, H., van de Welde, W., Wenzel, W., Wnek, J., Zhang, J.: The MONK’s Problems: A Performance Comparison of Different Learning Algorithms. Technical Report CMU-CS-91-197. Pittsburgh, PA.: Carnegie-Mellon University, Computer Science Department (1991)
Wanas, N.M., Hodge, L., Kamel, M.S.: Adaptive Training Algorithm for an Ensemble of Networks. In: Proceedings of the 2001 International Joint Conference on Neural Networks (IJCNN 2001), vol. 4, pp. 2590–2595. IEEE Computer Society Press, Los Alamitos (2001)
Wolberg, W.H., Mangasarian, O.L.: Multisurface Method of Pattern Separation for Medical Diagnosis Applied to Breast Cytology. Proceedings of the National Academy of Sciences, USA 87(23), 9193–9196 (1990)
Xu, L., Krzyzak, A., Suen, C.Y.: Several Methods for Combining Multiple Classifiers and Their Applications in Handwritten Character Recognition. IEEE Transactions on Systems, Man, and Cybernetics 22(3), 418–435 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Casey, M., Ahmad, K. (2004). In-Situ Learning in Multi-net Systems. In: Yang, Z.R., Yin, H., Everson, R.M. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2004. IDEAL 2004. Lecture Notes in Computer Science, vol 3177. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28651-6_112
Download citation
DOI: https://doi.org/10.1007/978-3-540-28651-6_112
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22881-3
Online ISBN: 978-3-540-28651-6
eBook Packages: Springer Book Archive