Using a general purpose meta neural network to adapt a parameter of the quickpropagation learning rule

Abstract

This paper proposes a refinement of an application independent method of automating learning rule parameter selection which uses a form of supervisor neural network, known as a Meta Neural Network, to alter the value of a learning rule parameter during training. The Meta Neural Network is trained using data generated by observing the training of a neural network and recording the effects of the selection of various parameter values. The Meta Neural Network is then combined with a normal learning rule to augment its performance. This paper investigates the combination of training sets for different Meta Neural Networks in order to improve the performance of a Meta Neural Network system. Experiments are undertaken to see how this method performs by using it to adapt a global parameter of the Quickpropagation learning rule.