Cited By
View all- Wilamowski BYu H(2010)Improved computation for Levenberg-Marquardt trainingIEEE Transactions on Neural Networks10.1109/TNN.2010.204565721:6(930-937)Online publication date: 1-Jun-2010
Efficient and reliable training of neural networks
Abstract
This paper introduces a neural network training tool, NBN 2.0, which is developed based on neuron by neuron computing method [1][2]. Error backpropagation (EBP) algorithm, Levenberg Marquardt (LM) algorithm and its improved versions are implemented in two different computing methods, traditional forward-backward computation and newly developed forward-only computation. The software can handle not only conventional multilayer perceptron (MLP) networks, but also arbitrarily connected neuron (ACN) networks. Several examples are presented to explain how to use this tool for neural network training. The software is developed based on Visual Studio platform using C++ language and it is available for everyone on the website.
References
[1]
B. M. Wilamowski, N. Cotton, J. Hewlett, O. Kaynak, "Neural network trainer with second order learning algorithms". Proc. International Conference on Intelligent Engineering Systems, June 29 2007-July 1 2007, pp. 127-132.
[2]
Wilamowski, B.M. Cotton, N.J. Kaynak, O. Dundar, G., "Computing Gradient Vector and Jacobian Matrix in Arbitrarily Connected Neural Networks", IEEE Trans. on Industrial Electronics, vol. 55, no. 10, pp. 3784-3790, Oct. 2008.
[3]
J. A. Farrell, M. M. Polycarpou, "Adaptive Approximation Based Control: Unifying Neural, Fuzzy and Traditional Adaptive Approximation Approaches, " IEEE Trans. on Neural Networks, vol. 19, no. 4, pp. 731-732, April 2008.
[4]
G. Colin, Y. Chamaillard, G. Bloch, G. Corde, "Neural Control of Fast Nonlinear Systems--Application to a Turbocharged SI Engine With VCT," IEEE Trans. on Neural Networks, vol. 18, no. 4, pp. 1101-1114, April 2007.
[5]
S. Khomfoi, L. M. Tolbert, "Fault diagnostic system for a multilevel inverter using a neural network". IEEE Trans. Power Electron., vol. 22, no. 3, pp. 1062-1069, May 2007.
[6]
J. F. Martins, V. Ferno Pires, A. J. Pires, "Unsupervised neural-network-based algorithm for an on-line diagnosis of three-phase induction motor stator fault". IEEE Trans. Ind. Electron., vol. 54, no. 1, pp. 259-264, Feb. 2007.
[7]
K. Cameron, A. Murray, "Minimizing the Effect of Process Mismatch in a Neuromorphic System Using Spike-Timing-Dependent Adaptation," IEEE Trans. on Neural Networks, vol. 19, no. 5, pp. 899-913, May 2008.
[8]
G. Indiveri, E. Chicca, R. Douglas, "A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity," IEEE Trans. on Neural Networks, vol. 17, no. 1, pp. 211-221, Jan 2006.
[9]
M. Kyperountas, A. Tefas, I. Pitas, "Weighted Piecewise LDA for Solving the Small Sample Size Problem in Face Verification," IEEE Trans. on Neural Networks, vol. 18, no. 2, pp. 506-519, Feb 2007.
[10]
Jafarzadegan, M., Mirzaei, H., "A new ensemble based classifier using feature transformation for hand recoognition", Human System Interactions, 2008 Conference on, pp. 749-754, May, 2008.
[11]
Mroczek, T., Paja, W., Piatek, L., Wrzesie, M., "Classification and synthesis of medical images in the domain of melanocytic skin lesions", Human System Interactions, 2008 Conference on, pp. 705-709, May, 2008.
[12]
Werbos P. J., "Back-propagation: Past and Future". Proceeding of International Conference on Neural Networks, San Diego, CA, 1, 343-354, 1988.
[13]
Yinyin Liu, J.A. Starzyk, Zhen Zhu, "Optimized Approximation Algorithm in Neural Networks Without Overfitting," IEEE Trans. on Neural Networks, vol. 19, no. 6, pp. 983-995, June 2008.
[14]
S. Ferrari, M. Jensenius, "A Constrained Optimization Approach to Preserving Prior Knowledge During Incremental Training," IEEE Trans. on Neural Networks, vol. 19, no. 6, pp. 996-1009, June 2008.
[15]
V.V. Phansalkar, P.S. Sastry, "Analysis of the back-propagation algorithm with momentum," IEEE Trans. on Neural Networks, vol. 5, no. 3, pp. 505-506, March 1994.
[16]
Fok Hing Chi Tivive, A. Bouzerdoum, "Efficient training algorithms for a class of shunting inhibitory convolutional neural networks," IEEE Trans. on Neural Networks, vol. 16, no. 3, pp. 541-556, March 2005.
[17]
M. Riedmiller, H. Braun, "A direct adaptive method for faster backpropagation learning: The RPROP algorithm". Proc. International Conference on Neural Networks, San Francisco, CA, 1993, pp. 586-591.
[18]
K. Levenberg, "A method for the solution of certain problems in least squares". Quarterly of Applied Machematics, 5, pp. 164-168, 1944.
[19]
A. Toledo, M. Pinzolas, J.J. Ibarrola, G. Lera, "Improvement of the neighborhood based Levenberg-Marquardt algorithm by local adaptation of the learning coefficient," IEEE Trans. on Neural Networks, vol. 16, no. 4, pp. 988-992, April 2005.
[20]
Bogdan M. Wilamowski, Hao Yu, "Neural Network Learning without Backpropagation". (unpublished)
[21]
J. R Alvarez-Sanchez, "Injecting knowledge into the solution of the two-spiral problem". Neural Compute and Applications, Vol. 8, pp. 265-272, 1999.
[22]
Wilamowski, B.M. Hunter, D. Malinowski, A., "Solving parity-N problems with feedforward neural networks". Proc. 2003 IEEE IJCNN, 2546-2551, IEEE Press, 2003.
Recommendations
Neural Network Trainer through Computer Networks
AINA '10: Proceedings of the 2010 24th IEEE International Conference on Advanced Information Networking and ApplicationsThis paper introduces a neural network training tool through computer networks. The following algorithms, such as neuron by neuron (NBN) [1][2], error back propagation (EBP), Levenberg Marquardt (LM) and its improved versions are implemented in two ...
An Efficient Algorithm for Complex-Valued Neural Networks Through Training Input Weights
Neural Information ProcessingAbstractComplex-valued neural network is a type of neural networks, which is extended from real number domain to complex number domain. Fully complex extreme learning machine (CELM) is an efficient algorithm, which owes faster convergence than the common ...
Comments
Information & Contributors
Information
Published In
May 2009
731 pages
ISBN:9781424439591
Sponsors
- Schneider Electric
- STMicroelectronics
- Iconics
Publisher
IEEE Press
Publication History
Published: 21 May 2009
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 21 Jan 2025
Other Metrics
Citations
Cited By
View all- Wilamowski BYu H(2010)Improved computation for Levenberg-Marquardt trainingIEEE Transactions on Neural Networks10.1109/TNN.2010.204565721:6(930-937)Online publication date: 1-Jun-2010