Abstract
This paper proposes a refinement of an application independent method of automating learning rule parameter selection which uses a form of supervisor neural network, known as a Meta Neural Network, to alter the value of a learning rule parameter during training. The Meta Neural Network is trained using data generated by observing the training of a neural network and recording the effects of the selection of various parameter values. The Meta Neural Network is then combined with a normal learning rule to augment its performance. This paper investigates the combination of training sets for different Meta Neural Networks in order to improve the performance of a Meta Neural Network system. Experiments are undertaken to see how this method performs by using it to adapt a global parameter of the Quickpropagation learning rule.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Schiffmann, W. Optimisation of the Backpropagation Algorithm for training multilayer perceptrons. Technical report, Dept of Physics, University of Kobelnz, Germany.ftp://archive.cis.ohio-state.edu/pub/neuroprose/schiff.bp_speedup.ps.Z (1993)
Riedmiller, M. Advanced supervised learning in multi-layered perceptrons: From Backpropagation to adaptive learning algorithms. Computer Standards and Interfaces. vol 16 part 3. 265–278. (1994)
Fahlman, S. Faster-Learning variations on Back-Propagation: An Empirical Study. Proceedings of the 1988 Connectionist Models Summer School. Morgan Kaufmann. (1988)
Prechelt, L. PROBEN1: A set of neural network benchmark problems and benchmarking rules. Technical Report 21/94, Dept. of Informatics, University of Karlsruhe, Germany. ftp://ftp.ira.uka.de/pub/neuron/proben1.tar.gz. (1994)
McCormack, C. A study of the Adaptation of Learning Rule Parameters using a Meta Neural Network. 13th European Meeting on Systems and Cybernetic Research, Vienna. Vol 2, 1043–1048. (1996)
McCormack, C. Parameter Adaptation using a Meta Neural Network. Proceedings of the World Conference on Neural Networks, Washington D.C., Vol 3, 147–150. (1995)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
McCormack, C. (1996). Using a general purpose meta neural network to adapt a parameter of the quickpropagation learning rule. In: von der Malsburg, C., von Seelen, W., Vorbrüggen, J.C., Sendhoff, B. (eds) Artificial Neural Networks — ICANN 96. ICANN 1996. Lecture Notes in Computer Science, vol 1112. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61510-5_83
Download citation
DOI: https://doi.org/10.1007/3-540-61510-5_83
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61510-1
Online ISBN: 978-3-540-68684-2
eBook Packages: Springer Book Archive