Abstract
A gradient-based algorithm for ensemble weights modification is presented and applied on the regression tasks. Simulation results show that this method can produce an estimator ensemble with better generalization than those of bagging and single neural network. The method can not only have a similar function to GASEN of selecting many subnets from all trained networks, but also be of better performance than GASEN, bagging and best individual of regressive estimators.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Optiz, D., Shavlik, J.: Actively Searching For an Effectively Neural Network Ensemble. Connection Science 8(3-4), 337–353 (1996)
Hanson, L.K., Salamon, P.: Neural Network Ensemble. IEEE Trans. on Pattern Analysis and Machine Intelligence PAMA-12, 993–1002 (1990)
Breiman, L.: Bagging Predictors. Machine Leaning 24, 123–140 (1996)
Schapire, R.E.: The Strength of Weak Learn Ability. Machine Leaning 5, 197–227 (1990)
Freund, Y.: Boosting a Weak Algorithm by Majority. Information Computation 121, 256–285 (1995)
Freund, Y., Schapire, R.E.: A Decision-theoretic Generalization of On-Line Learning and an Application to Boosting. J. Computer and System Science 55, 119–139 (1997)
Drucker, H.: Improving Regressors Using Boosting Techniques. In: Proc. of 14th International Conf. on Machine Learning, pp. 107–115. Morgan Kaufmann, Burlington, MA (1997)
Schapire, R.E., Singer, Y.: Improved Boosting Algorithms Using Confidence-rated Predictions. Machine Learning 37(3), 297–336 (1999)
Avnimilach, R., Intrator, N.: Boosted Mixture of Experts: An Ensemble Learning Scheme. Neural Computation 11, 483–497 (1999)
Solomatine, D.P., Shrestha, D.L.: AdaBoost.RT: a boosting algorithm for regression problems. In: Proc. of 2004 IEEE International Joint Conf. on Neural Networks, vol. 2, pp. 1163–1168 (2004)
Islam, M., Yao, X., Murase, K.: A Constructive Algorithm for Training Cooperative Neural Network Ensembles. IEEE Trans. on Neural Networks 14(4), 820–834 (2003)
Liu, Y., Yao, X.: Simultaneous Training of Negatively Correlated Neural Networks in an Ensemble. IEEE Trans. on System, Man and Cybernetics—PART B: Cybernetics 29(6), 716–725 (1999)
Jang, M., Cho, S.: Observational Learning Algorithm for an Ensemble of Neural Networks. Pattern Analysis & Application 5, 154–167 (2002)
Perrone, M.P., Cooper, L.N.: When Networks Disagree: Ensemble Method for Neural Networks. In: Artificial Neural Networks for Speech and Vision, pp. 126–142. Chapman and Hall, New York (1993)
Zhou, Z.H., Wu, J.X., Tang, W.: Ensembling Neural Networks: Many Could Be Better Than All. Artificial Intelligence 137, 239–263 (2002)
Zhou, Z.H., Wu, J.X., Jiang, Y., et al.: Genetic Algorithm based Selective Neural Network Ensemble. In: Proc. 17th International Joint Conf. on Artificial Intelligence, Seattle, WA, vol. 2, pp. 797–802 (2001)
German, S., Bienenstock, E., Doursat, R.: Neural Networks And the Bias/Variance Dilemma. Neural Computation 4(1), 1–58 (1992)
Kroph, A., Vedelsby, J.: Neural Network Ensemble, Cross Validation, and Active Learning. In: Advanced in Neural Information Processing System, vol. 7, pp. 231–238. MIT Press, Cambridge (1995)
Friedman, J.: Multivariate Adaptive Regression Splines. Annals of Statistics 19, 1–141 (1991)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Meng, J., An, K. (2006). Aggregating Regressive Estimators: Gradient-Based Neural Network Ensemble. In: Gelbukh, A., Reyes-Garcia, C.A. (eds) MICAI 2006: Advances in Artificial Intelligence. MICAI 2006. Lecture Notes in Computer Science(), vol 4293. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11925231_30
Download citation
DOI: https://doi.org/10.1007/11925231_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-49026-5
Online ISBN: 978-3-540-49058-6
eBook Packages: Computer ScienceComputer Science (R0)