Abstract
There exists a substantial problem in obtaining good generalisation performance in the application of artificial neural network technology where training data is limited. Generalisation ability is analysed for a number of computational paradigms which attempt to alleviate this for the multilayer perceptron. The problem of line detection in a time/frequency sonar image or ‘lofargram’ is adopted as a case study on which to assess these techniques. The effect on neural network generalisation performance is studied for (a) heuristically changing the number of hidden nodes, (b) weight decay, (c) soft weight-sharing, and (d) Ockham's networks. These techniques are introduced from the perspective of the Minimum Description Length principle. Results show that the use of weight decay and Ockham's networks are able to improve generalisation beyond that available by simply altering the number of hidden nodes. It is shown that line detection in lofargram images is possible at a success rate of 85% for data outside of the training set.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Rumelhart DE, Hinton GE, Williams RJ. Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL (eds). Parallel Distributed Processing. Cambridge (MA): MIT Press, Vol. 1, 1986
Johansson EM, Dowla FU, Goodman DM. Backpropagation learning for multilayer feed-forward neural networks using the conjugate gradient method, Int J Neural Syst 1992; 2(4): 291–301
Rissanen J. Stochastic Complexity in Statistical Inquiry. New York (NY): World Scientific, 1989
Frean M. The upstart algorithm. Neural Computation 1990; 2(2): 198–209
Fahlman SE, Lebiere C. The cascade-correlation learning architecture. In: Tourzetsky DS (ed). Advances in Neural Information Processing Systems 2. San Meteo (CA): Morgan Kaufmann, 1990, 524–532
Hanson SJ, Pratt LY. Comparing biases for minimal network construction with back-propagation. In: Tourzetsky DS (ed). Advances in Neural Information Processing Systems 1. San Meteo (CA): Morgan Kaufmann, 1989, 177–185
Weigend DH, Rumelhart DE, Huberman BA. Generalisation by weight-elimination with application to forecasting. In: Lipmann RP, Moody JE, Tourzetsky DS (eds). Advances in Neural Information Processing Systems 3. San Meteo (CA): Morgan Kaufmann, 1991, 875–882
Krogh A, Hertz JA. A simple weight decay can improve generalization. In: Lipmann RP, Moody JE, Tourzetsky DS (eds). Advances in Neural Information Processing Systems 4, San Meteo (CA): Morgan Kaufmann, 1992, 950–957
MacKay DJC. A practical Bayesian framework for backprop networks. Neural Computation, 1992; 4(3): 448–472
Nowlan SJ, Hinton GE. Simplifying neural networks by soft weight-sharing. Neural Computation, 1992; 4(4): 473–493
Kendall GD, Hall TJ. Optimal network construction by Minimum Description Length. Neural Computation, 1993; 5(2): 210–212
Kendall GD, Hall TJ. Ockham's nets: Self-adaptive minimal neural networks. In: Aleksander I, Taylor JG (eds). Artificial Neural Networks 2. Amsterdam: North-Holland, 1992, 183–186
Smith DJ, Harris JI. Line tracking using artificial neural networks and fuzzy inference. In: Undersea Defence Technology Conference Proceedings, Microwave Exhibitions and Publishers, 1991
Press WH, Flannery BP, Teukolsky SA, Vetterling WT. Numerical recipes: the art of scientific computing. Cambridge: Cambridge University Press, 1986
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kendall, G.D., Hall, T.J. & Newton, T.J. An investigation of the generalisation performance of neural networks applied to lofargram classification. Neural Comput & Applic 1, 147–159 (1993). https://doi.org/10.1007/BF01414434
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01414434