research-article

Learning Bias in Neural Networks and an Approach to Controlling Its Effect in Monotonic Classification

Authors:

S. WangAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 15, Issue 9

Pages 962 - 966

https://doi.org/10.1109/34.232084

Published: 01 September 1993 Publication History

Abstract

As a learning machine, a neural network using the backpropagation training algorithm is subject to learning bias. This results in unpredictability of boundary generation behavior in pattern recognition applications, especially in the case of small training sample size. It is suggested that in a large class of pattern recognition problems such as managerial and other problems possessing monotonicity properties, the effect of learning bias can be controlled by using multiarchitecture monotonic function neural networks.

References

[1]

{1} E. Barnard and D. Casasent, "A comparison between criterion functions for linear classifiers, with an application to neural nets," <i>IEEE Trans. Syst. Man Cybern.</i>, vol. 19, no. 5, pp. 1030-1041, 1989.]]

[2]

{2} T. Kohonen, G. Barna, and R. Chrisley, "Statistical pattern recognition with neural networks: Benchmarking studies," in <i>Proc. IEEE Int. Conf. Neural Networks</i>, (San Diego), 1988, pp. I:61-68.]]

[3]

{3} D. F. Specht, "Probabilistic neural networks," <i>Neural Networks</i>, vol. 3, no. 1, pp. 109-118, 1990.]]

Digital Library

[4]

{4} D. F. Specht, "Probabilistic neural networks and the polynomial adaline as complementary techniques for classification," <i>IEEE Trans. Neural Networks</i>, vol. 1, no. 1, pp. 111-121, 1990.]]

Digital Library

[5]

{5} T. Kawabata, "Generalization effects of k-neighbor interpolation training," <i>Neural Comput.</i>, vol. 3, pp. 409-417, 1991.]]

[6]

{6} D. P. Casasent and E. Barnard, "Adaptive-clustering optical neural net," <i>Applied Optics</i>, vol. 29, pp. 2603-2615, 1990.]]

[7]

{7} D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, <i>Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations</i>. Cambridge, MA: MIT Press, 1986.]]

Digital Library

[8]

{8} D. J. Hand, <i>Discrimination and Classification</i>. New York: Wiley, 1981.]]

[9]

{9} F. F. Soulie, P. Gallinari, Y. L. Cun, and S. Thiria, "Evaluation of network architectures on test learning tasks," in <i>Proc. IEEE First Int. Conf. Neural Networks</i> (San Diego), 1987, pp. II:653-660.]]

[10]

{10} P. E. Utgoff, "Shift of bias for inductive concept learning," in <i>Machine Learning: An Artificial Intelligence Approach</i> (R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Eds.). Los Altos, CA: Morgan Kaufman 1986, pp. 107-148, vol. II.]]

[11]

{11} L. Rendell, R. Seshu, and D. Tcheng, "More robust concept learning using dynamically-variable bias," in <i>Proc. Fourth Int. Workshop Machine Learning</i> (M. B. Morgan, Ed.). Los Altos, CA: Morgan Kaufmann, pp. 66-78, 1987.]]

[12]

{12} G. Cybenko, "Approximation by superpositions of a sigmoidal function," <i>Math. Contr. Signals Syst.</i>, vol. 2, no. 4, pp. 303-314, 1989.]]

[13]

{13} P. E. Utgoff, <i>Machine Learning Of Inductive Bias</i>. Boston, MA: Kluwer, 1986.]]

Digital Library

[14]

{14} E. I. Altman, "Financial ratios, discriminant analysis and the prediction of corporate bankruptcy," <i>J. Finance</i>, vol. 28, pp. 589-609, 1968.]]

[15]

{15} C. C. Greer, "Deciding to accept or reject a marginal retail credit application," <i>J. Retailing</i>, vol. 43, pp. 44-53, 1968.]]

[16]

{16} V. Ramanujam, N. Venkatraman, and J. C. Camillus, "Multiobjective assessment of effectiveness of strategic planning: A discriminant analysis approach," <i>Acad Management J.</i>, vol. 29, pp. 347-372, 1986.]]

[17]

{17} R. H. Keeney and H. Raiffa, <i>Decisions with Multiple Objectives: Preferences and Value Tradeoffs</i>. New York: Wiley, 1976.]]

[18]

{18} R. A. Fisher, "The use of multiple measurements in taxonomic problems," <i>Ann. Eugenics</i>, vol. 7, pp. 179-188, 1936.]]

[19]

{19} Y. T. Chien and T. J. Killeen, "Computer and statistical considerations for oil spill identification," in <i>Handbook of Statistics</i>, (P. R. Krishnaiah and L. N. Kanel, Eds.). New York: North Holland, 1982, pp. 651-671, vol. 2.]]

[20]

{20} N. P. Archer and S. Wang, "Application of the back propagation neural network algorithm with monotonicity constraints for two-group classification problems," <i>Decision Sci.</i>, vol. 24, pp. 60-75, 1993.]]

[21]

{21} A. Wieland and R. Leighton, "Geometric analysis of neural network capabilities," in <i>Proc. IEEE First Int. Conf. Neural Network</i> (San Diego) 1987, pp. III:385-392.]]

Cited By

Moudani WZaarour GMora-Camino F(2018)Fuzzy Prediction of Insolvent Customers in Mobile TelecommunicationInternational Journal of Strategic Information Technology and Applications10.4018/ijsita.20140701015:3(1-23)Online publication date: 13-Dec-2018
https://dl.acm.org/doi/10.4018/ijsita.2014070101
Moudani WZaarour GMora-Camino F(2018)Fuzzy Classification of Customer Insolvency in Mobile TelecommunicationInternational Journal of Decision Support System Technology10.4018/ijdsst.20140701016:3(1-29)Online publication date: 17-Dec-2018
https://dl.acm.org/doi/10.4018/ijdsst.2014070101
Daniels HVelikova M(2010)Monotone and partially monotone neural networksIEEE Transactions on Neural Networks10.1109/TNN.2010.204480321:6(906-917)Online publication date: 1-Jun-2010
https://dl.acm.org/doi/10.1109/TNN.2010.2044803

Index Terms

Learning Bias in Neural Networks and an Approach to Controlling Its Effect in Monotonic Classification
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

On the Problem of Local Minima in Backpropagation

The authors propose a theoretical framework for backpropagation (BP) in order to identify some of its limitations as a general learning procedure and the reasons for its success in several experiments on pattern recognition. The first important ...
Comparative Analysis of Backpropagation and the Extended Kalman Filter for Training Multilayer Perceptrons

The relationship between backpropagation and extended Kalman filtering for training multilayer perceptrons is examined. These two techniques are compared theoretically and empirically using sensor imagery. Backpropagation is a technique from neural ...
Apnea recognition using neural networks

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 15, Issue 9

September 1993

125 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © Copyright © 1993 IEEE. All Rights Reserved.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 September 1993

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Moudani WZaarour GMora-Camino F(2018)Fuzzy Prediction of Insolvent Customers in Mobile TelecommunicationInternational Journal of Strategic Information Technology and Applications10.4018/ijsita.20140701015:3(1-23)Online publication date: 13-Dec-2018
https://dl.acm.org/doi/10.4018/ijsita.2014070101
Moudani WZaarour GMora-Camino F(2018)Fuzzy Classification of Customer Insolvency in Mobile TelecommunicationInternational Journal of Decision Support System Technology10.4018/ijdsst.20140701016:3(1-29)Online publication date: 17-Dec-2018
https://dl.acm.org/doi/10.4018/ijdsst.2014070101
Daniels HVelikova M(2010)Monotone and partially monotone neural networksIEEE Transactions on Neural Networks10.1109/TNN.2010.204480321:6(906-917)Online publication date: 1-Jun-2010
https://dl.acm.org/doi/10.1109/TNN.2010.2044803

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents