Abstract
Consider a multilayer perceptron (MLP) with d inputs, a single hidden sigmoidal layer and a linear output. By adding an additional d inputs to the network with values set to the square of the first d inputs, properties reminiscent of higher-order neural networks and radial basis function networks (RBFN) are added to the architecture with little added expense in terms of weight requirements. Of particular interest, this architecture has the ability to form localized features in a d-dimensional space with a single hidden node but can also span large volumes of the input space; thus, the architecture has the localized properties of an RBFN but does not su_er as badly from the curse of dimensionality. I refer to a network of this type as a SQuare Unit Augmented, Radially Extended, MultiLayer Perceptron (SQUARE-MLP or SMLP).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
M. Casdagli. Nonlinear prediction of chaotic time series. Physica D, 35:335–356, 1989.
D. H. Deterding. Speaker Normalisation for Automatic Speech Recognition. PhD thesis, University of Cambridge, 1989.
S.E. Fahlman. Faster-learning variations on back-propagation: An empirical study. In Proceedings of the 1988 Connectionist Models Summer School. Morgan Kaufmann, 1988.
S.E. Fahlman and C. Lebiere. The cascade-correlation learning architecture. In S. Touretzky, editor, Advances in Neural Information Processing Systems 2. Morgan Kaufmann, 1990.
M. Finke and K.-R. Müller. Estimating a-posteriori probabilities using stochastic network models. In M. Mozer, P. Smolensky, D.S. Touretzky, J.L. Elman, and A.S. Weigend, editors, Proceedings of the 1993 Connectionist Models summer school, pages 324–331, Hillsdale, NJ, 1994. Erlenbaum Associates.
T. Hastie and R. Tibshirani. Flexible discriminant analysis by optimal scoring. Technical report, AT&T Bell Labs, Murray Hill, New Jersey, 1993.
T. Hastie and R. Tibshirani. Discriminant adaptive nearest neighbor classifcation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(6):607–616, June 1996.
S. Hochreiter and J. Schmidhuber. Lococode. Technical Report FKI-222-97, Fakult Ät für Informatik, Technische UniversitÄt München, 1997.
K. J. Lang and M. J. Witbrock. Learning to tell two spirals apart. In Proceedings of the 1988 Connectionist Models Summer School. Morgan Kaufmann, 1988.
A. Lapedes and R. Farber. Nonlinear signal processing using neural networks: Prediction and system modelling. Technical Report LA-UR-87-2662, Los Alamos National Laboratory, Los Alamos, NM, 1987.
A. Lapedes and R. Farber. How neural nets work. In D.Z. Anderson, editor, Neural Information Processing Sysytems, pages 442–456. American Institute of Physics, New York, 1988.
S. Lawrence, A. C. Tsoi, and A. D. Back. Function approximation with neural networks and local methods: Bias, variance and smoothness. In Peter Bartlett, Anthony Burkitt, and Robert Williamson, editors, Australian Conference on Neural Networks, pages 16–21. Australian National University, 1996.
S. Lee and R.M. Kil. Multilayer feedforward potential function networks. In IEEE international Conference on Neural Networks, pages I:161–171. San Diego: SOS Printing, 1988.
Y.C. Lee, G. Doolen, H.H. Chen, G.Z. Sun, T. Maxwell, H.Y. Lee, and C.L. Giles. Machine learning using higher order correlation networks. Physica D, 22-D:276–306, 1986.
J. Moody and C. Darken. Learning with localized receptivefields. In D. Touretsky, G. Hinton, and T. Sejnowski, editors, Proceedings of the 1988 Connectionist Models Summer School. Morgan-Kaufmann, 1988.
J. Moody and C. Darken. Fast learning in networks of locally-tuned processing units. Neural Computation, 1:281–294, 1989.
M. Niranjan and F. Fallside. Neural networks and radial basis functions in classifying static speech patterns. Computer Speech and Language, 4(275–289), 1990.
Y.H. Pao. Adaptive Pattern Recognition and Neural Networks. Addison-Wesley Publishing Company, Inc., Reading, Massachusetts, 1989.
A. J. Robinson. Dynamic Error Propagation Networks. PhD thesis, Cambridge University, 1989.
D.E. Rumelhart, J.L. McClelland, and the PDP Research Group. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. The MIT Press., 1986. 2 vols.
W. Sarle. The comp.ai.neural-nets Frequently Asked Questions List, 1997.
M. Schetzen. The Volterra and Wiener Theories of Nonlinear Systems. John Wiley and Sons, New York, 1980.
B. Schölkopf, A. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Technical report, Max-Planck-Institut für biologische Kybernetik, 1996. and Neural Computation, 10, 5, 1299–1319. 1998.
V. Volterra. Theory of Functionals and of Integro-di_erential Equations. Dover, 1959.
P. Werbos. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University, 1974.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Flake, G.W. (1998). Square Unit Augmented Radially Extended Multilayer Perceptrons. In: Orr, G.B., Müller, KR. (eds) Neural Networks: Tricks of the Trade. Lecture Notes in Computer Science, vol 1524. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49430-8_8
Download citation
DOI: https://doi.org/10.1007/3-540-49430-8_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65311-0
Online ISBN: 978-3-540-49430-0
eBook Packages: Springer Book Archive