article

A Fast Simplified Fuzzy ARTMAP Network

Authors:

Mohammad-Taghi Vakil-Baghmisheh,

Nikola PavešićAuthors Info & Claims

Neural Processing Letters, Volume 17, Issue 3

Pages 273 - 316

https://doi.org/10.1023/A:1026004816362

Published: 01 June 2003 Publication History

Publisher Site

Abstract

We present an algorithmic variant of the simplified fuzzy ARTMAP (SFAM) network, whose structure resembles those of feed-forward networks. Its difference with Kasuba's model is discussed, and their performances are compared on two benchmarks. We show that our algorithm is much faster than Kasuba's algorithm, and by increasing the number of training samples, the difference in speed grows enormously.

The performances of the SFAM and the MLP (multilayer perceptron) are compared on three problems: the two benchmarks, and the Farsi optical character recognition (OCR) problem. For training the MLP two different variants of the backpropagation algorithm are used: the BPLRF algorithm (backpropagation with plummeting learning rate factor) for the benchmarks, and the BST algorithm (backpropagation with selective training) for the Farsi OCR problem.

The results obtained on all of the three case studies with the MLP and the SFAM, embedded in their customized systems, show that the SFAM's convergence in fast-training mode, is faster than that of MLP, and online operation of the MLP is faster than that of the SFAM. On the benchmark problems the MLP has much better recognition rate than the SFAM. On the Farsi OCR problem, the recognition error of the SFAM is higher than that of the MLP on ill-engineered datasets, but equal on well-engineered ones. The flexible configuration of the SFAM, i.e. its capability to increase the size of the network in order to learn new patterns, as well as its simple parameter adjustment, remain unchallenged by the MLP.

References

[1]

1. McCulloch, W. S. and Pitts, W.: A logical calculus of the ideas imminent in nervous activity, Bulletin of Mathematical Biophysics, 5 (1943), 115-133.

Crossref

Google Scholar

[2]

2. Grossberg, S.: Adaptive pattern classification and universal recoding, II: feedback, expectation, olfaction and illusions, Biological Cybernetics, 23 (1976), 187-202.

Digital Library

Google Scholar

[3]

3. Carpenter, G. A. and Grossberg, S.: A massively parallel architecture for a self organizing neural pattern recognition machine, Computer Vision, Graphics and Image Processing, 37 (1987), 54-115.

Digital Library

Google Scholar

[4]

4. Carpenter, G. A. and Grossberg, S.: ART2: Self-organization of stable category recognition codes for analog input patterns, Applied Optics, 26(23) (1987), 4919- 4930.

Crossref

Google Scholar

[5]

5. Carpenter, G. A. and Grossberg, S.: ART3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architecture, Neural Networks, 3 (1990), 129- 152.

Crossref

Google Scholar

[6]

6. Carpenter, G. A., Grossberg G. S. and Reynolds J. H.: ARTMAP: Supervised real-time learning and classification of mon-stationary data by a self-organizing neural network, Neural Networks, 4 (1991), 565-588.

Digital Library

Google Scholar

[7]

7. Carpenter, G. A., Grossberg, S. and Rosen, D. B.: Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system, Neural Networks, 4 (1991), 759-771.

Digital Library

Google Scholar

[8]

8. Carpenter, G. A., Grossberg, S., Markuzon, N., Reynolds, J. H. and Rosen, D. B.: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps, IEEE Trans. on Neural Networks, 3(5) (1992), 698-713.

Digital Library

Google Scholar

[9]

9. Carpenter, G. A., Grossberg, S. and Rosen, D. B.: ART2-A: An Adaptive Resonance Algorithm for Rapid Category Learning and Recognition, Neural Networks, 4, (1991), 493-504.

Digital Library

Google Scholar

[10]

10. Carpenter, G. A. and Grossberg, S.: A self-organizing neural network for supervised learning, recognition, and prediction, IEEE Communications Mag., pp. 38-49, Sep. 1992.

Digital Library

Google Scholar

[11]

11. Kasuba, T.: Simplified Fuzzy ARTMAP, AI Expert, 8(11) (1993), 18-25.

Google Scholar

[12]

12. Marriott, S. and Harrison, R. F.: A modified fuzzy ARTMAP architecture for the approximation of noisy mappings, Neural-Networks, 8(4) (1995), 619-641.

Digital Library

Google Scholar

[13]

13. Malkani, A. and Vassiliadis, C. A.: Parallel implementation of the fuzzy ARTMAP neural network paradigm on a hypercube, Expert Systems, 12(1) (1995), 39-53.

Crossref

Google Scholar

[14]

14. Chen, B. and Hoberock, L. L.: A Fuzzy Neural Network Architecture for Fuzzy Control and Classification, ICNN 96, The 1996 IEEE International Conference on Neural Networks (Cat. No. 96CH35907), IEEE, 2, pp. 1168-1173 New York, USA, 1996.

Google Scholar

[15]

15. Blume, M., Van-Blerkom, D. A. and Esener, S. C.: Fuzzy ARTMAP Modifications for Intersecting Class Distributions, WCNN'96, World Congress on Neural Networks, International Neural Network Society 1996 Annual Meeting. pp. 250-255. Lawrence Erlbaum Assoc, USA, 1996.

Google Scholar

[16]

16. Chee, P. L. and Harrison, R. F.: Modified fuzzy ARTMAP approaches bayes optimal classification rates: An empirical demonstration, Neural Networks, 10(4) (1997), 755-774.

Digital Library

Google Scholar

[17]

17. Srinivasa, N.: Learning and generalization of noisy mappings using a modified PROBART neural network, IEEE Trans. Sign. Proc., 45(10) (1997), 2533-2550.

Digital Library

Google Scholar

[18]

18. Hsien, L. T. and Shie, J. L.: A Neural Network Model for Spoken Word Recognition, IEEE International Conference on Systems, Man, and Cybernetics, Computational Cybernetics and Simulation (Cat. no. 97CH36088-5), IEEE, 5, pp. 4029-4034 New York, USA, 1997.

Crossref

Google Scholar

[19]

19. Carpenter, G. A. and Markuzon, N.: ARTMAP-IC and medical diagnosis: instance counting and inconsistent cases, Neural Networks, 11(2) (1998), 323-336.

Digital Library

Google Scholar

[20]

20. Healy, M. J. and Caudell, T. P.: Guaranteed two-Pass convergence for supervised and inferential learning, IEEE Tran. Neural Networks, 9(1) (1998), 195-204.

Digital Library

Google Scholar

[21]

21. Jervis, B. W., Garcia, T. and Giahnakis, E. P.: Probabilistic simplified fuzzy ARTMAP (PSFAM) and application to biosignal data, IEE Proc. Science Measur. and Tech., 146(4) (1999), 165-169.

Crossref

Google Scholar

[22]

22. Verzi, S. J., Heileman, G. L., Georgiopoulos, M. and Healy, M. J.: Boosted ARTMAP, IEEE International Joint Conference on Neural Networks Proce. IEEE World Congress on Computational Intelligence (Cat. no. 98CH36227), IEEE, 1, pp. 396-401 New York, USA, 1998.

Crossref

Google Scholar

[23]

23. Dagher, I., Georgiopoulos, M., Heileman, G. L. and Bebis, G.: Fuzzy ARTVar: An Improved Fuzzy ARTMAP Algorithm, IEEE International Joint Conference on Neural Networks Proc. IEEE World Congress on Computational Intelligence (Cat. no. 98CH36227). IEEE, 3, pp. 1688-1693 New York, USA, 1998.

Crossref

Google Scholar

[24]

24. Werbos, P. J.: Beyond Regression: New Tools for Prediction and Analysis in Behavioral Science, Ph.D Thesis, Harvard University, Cambridge, MA., 1974.

Google Scholar

[25]

25. Parker, D.B. Learning Logic, Invention Report s81-64, File 1, Office of Technology Licensing, Stanford University, March 1982.

Google Scholar

[26]

26. Rumelhart, D. E. and McClelland, J. L.: Parallel Distributed Processing: Exploration in the Microstructure of Cognition, MIT press, 1986.

Digital Library

Google Scholar

[27]

27. Jondarr, G.: Backpropagation Family Album, Technical Report C/TR96-05, Department of Computing, Macquarie University, New South Wales, August, 1996.

Google Scholar

[28]

28. Aggarwal, R. K., Xuan, Q. Y., Johns, A. T., Furong, L. I. and Bennett, A.: A novel approach to fault diagnosis in multicircuit transmission lines using fuzzy ARTMAP neural networks, IEEE Trans. on Neural networks, 10(5) (1999), 1214-1221.

Digital Library

Google Scholar

[29]

29. Gardner, J. W., Shin, H. W., Hines, E. L. and Dow, C. S.: An electronic nose system for monitoring the quality of potable water, Sensors and Actuators B (Chemical), B69(3) (2000), 336-341.

Crossref

Google Scholar

[30]

30. Heinke, D. and Hamker, F. H.: Comparing neural networks: a benchmark on growing neural gas, growing cell structures, and fuzzy ARTMAP, IEEE Trans. on Neural Networks , 9(6) (1998), 1279-1291.

Digital Library

Google Scholar

[31]

31. Hines, W. L., Llobet, E. and Gardner, J. W.: Neural network based electronic nose for apple ripeness determination, Electronics Letters, 35(10) (1999), 821-823.

Crossref

Google Scholar

[32]

32. Keyvan, S., Xiaolong, S. and Kell, M.: Nuclear fuel, pellet inspection using artificial neural networks, Journal of Nuclear Materials, 264(12) (1999), 141-154.

Crossref

Google Scholar

[33]

33. Liobet, E., Hines, E. L., Gardner, J. W., Bartlett, P. N. and Mottram, T. T.: Fuzzy ARTMAP based electronic nose data analysis, Sensors and Actuators B (Chemical), B61(13) (1999), 183-190.

Crossref

Google Scholar

[34]

34. Ludwig, L., Sapozhnikova, E., Lunin, V. and Rosenstiel, W.: Error classification and yield prediction of chips in semiconductor industry applications, Neural Computing & Applications , 9(3) (2000), 202-210.

Crossref

Google Scholar

[35]

35. Mills, H., Burton, D. R. and Lalor, M. J.: Applying backpropagation neural networks to fringe analysis, Optics and Lasers in Engineering, 23(5) (1995), 33-41.

Crossref

Google Scholar

[36]

36. Muchoney, D. and Williamson, J.: A Gaussian adaptive resonance theory neural network classification algorithm applied to supervised land cover mapping using multitemporal vegetation index data, IEEE transactions on Geoscience and Remote Sensing, 39(9) (2001), 1969-1977.

Google Scholar

[37]

37. Obaidat, M. S., Khalid, H. and Sadoun, B.: Ultrasonic transducer characterization by neural networks, Information Sciences, 107(14) (1998), 195-215.

Digital Library

Google Scholar

[38]

38. Obaidat, M. S. and Sadoun, B.: Verification of computer users using keystroke dynamics, IEEE Trans. on Systems, Man and Cybernetics, Part B (Cybernetics), 27(2) (1997), 261- 269.

Digital Library

Google Scholar

[39]

39. Raveendran, P., Palaniappan, R. and Omatu, S.: Fuzzy ARTMAP classification of invariant features derived using angle of rotation from a neural network, Information Sciences, 130(14) (2000), 67-84.

Digital Library

Google Scholar

[40]

40. Shin, H. W., Llobet, E., Gardner, J. W., Hines, E. L. and Dow, C. S.: Classification of the strain and growth phase of cyanobacteria in potable water using an electronic nose system, IEE Proc. Science, Measurement and Technology, 147(4) (2000), 158-164.

Crossref

Google Scholar

[41]

41. Walczak, S.: Neural network models for a resource allocation problem, IEEE Trans. on Systems, Man and Cybernetics, Part B (Cybernetics), 28(2) (1998), 276-284.

Digital Library

Google Scholar

[42]

42. Song, X. H., Hopke, P. K., Bruns, M. A., Bossio, D. A. and Scow, K. M.: A fuzzy adaptive resonance theory supervised predictive mapping neural network applied to the classification of multivariate chemical data, Chemometrics and Intelligent Laboratory Systems, 41(2) (1998), 161-170.

Crossref

Google Scholar

[43]

43. Zhang, Z. and Manikopoulos, C.: Neural networks in statistical anomaly intrusion detection, Neural Network World, 11(3) (2001), 305-316.

Google Scholar

[44]

44. Carpenter, G. A., Gjaja, M. N., Gopal, S. and Woodcock, C. E.: ART Neural Networks for Remote Sensing: Vegetation Classification from Landsat TM and Terrain Data, IEEE Trans. on Geoscience and Remote Sensing, 39(2) (1997), 308-325.

Crossref

Google Scholar

[45]

45. Rojas, R.: Neural Networks: A Systematic Introduction, New York, Springer-Verlag, 1996.

Digital Library

Google Scholar

[46]

46. Fahlman, S. C. and Lebiere, C.: The cascade-correlation learning architecture, Technical Report, CMU-CS-90-100, Carnegie Mellon University, Pittsburgh, 1988.

Google Scholar

[47]

47. Fahlman, S. E.: An empirical study of learning speed in backpropagation networks, Technical Report, CMU-CS-88-162, Carnegie Mellon University, Pittsburgh, 1988.

Google Scholar

[48]

48. Haykin, S.: Neural Networks: A Comprehensive Foundation, USA, Macmillan, 1994.

Digital Library

Google Scholar

[49]

49. Theodoridis, S. and Koutroumbas, K. Pattern Recognition, USA, Academic Press, 1999.

Google Scholar

[50]

50. Vakil, M.T. and Pavesic, N. Backpropagation with declining learning rate, In: Proc. of the 10th Electrotechnical and Computer Science Conference, pp. 297-300. Portoroz, Slovenia, B, 2001.

Google Scholar

[51]

51. Vakil, M.-T.: Farsi Character Recognition Using Artificial Neural Networks, Ph.D Thesis, Faculty of Electrical Engineering, University of Ljubljana, October 2002.

Google Scholar

[52]

52. Diamantaras, K. I. and Kung, S. Y.: Principal Component Neural Networks: Theory and Applications, JohnWiley & Sons, 1996.

Digital Library

Google Scholar

Cited By

View all

Bakirov RFay DGabrys B(2021)Automated adaptation strategies for stream learningMachine Language10.1007/s10994-021-05992-x110:6(1429-1462)Online publication date: 1-Jun-2021
https://dl.acm.org/doi/10.1007/s10994-021-05992-x
Brito da Silva LElnabarawy IWunsch D(2019)A survey of adaptive resonance theory neural network models for engineering applicationsNeural Networks10.1016/j.neunet.2019.09.012120:C(167-203)Online publication date: 1-Dec-2019
https://dl.acm.org/doi/10.1016/j.neunet.2019.09.012
de Oliveira MInman D(2017)Performance analysis of simplified Fuzzy ARTMAP and Probabilistic Neural Networks for identifying structural damage growthApplied Soft Computing10.1016/j.asoc.2016.12.02052:C(53-63)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.1016/j.asoc.2016.12.020
Show More Cited By

Index Terms

A Fast Simplified Fuzzy ARTMAP Network

Recommendations

Training RBF networks with selective backpropagation

Backpropagation with selective training (BST) is applied on training radial basis function (RBF) networks. It improves the performance of the RBF network substantially, in terms of convergence speed and recognition error. Three drawbacks of the basic ...
Clearly defined architectures of neural networks and multilayer perceptron

Neural networks with clearly defined architecture differ in the fact that they make it possible to determine the structure of neural network (number of neurons, layers, connections) on the basis of initial parameters of recognition problem. For these ...
Electric load forecasting using a fuzzy ART&ARTMAP neural network

This work presents a neural network based on the ART architecture (adaptive resonance theory), named fuzzy ART&ARTMAP neural network, applied to the electric load-forecasting problem. The neural networks based on the ART architecture have two ...

Reviews

Reviewer: Ramaswamy Palaniappan

The authors of this paper present a modification to the simplified fuzzy ARTMAP (SFAM) neural network (NN) proposed by Kasuba. The modified SFAM algorithm is faster to train, and gives improved recognition accuracy over Kasuba's SFAM for two benchmarks and a Farsi optical character recognition (OCR) problem. The main innovation lies in using the activity level of uncommitted neurons as a training parameter. The reduction in recognition error rate is the important contribution of the modified SFAM, while the slight improvements in training time may not be a significant advantage, especially with the current improvements in computer processing power, and when used on an average dataset size. The authors have also compared the performances of their modified SFAM with multilayer perceptron (MLP), trained using two different algorithms, on these three datasets. MLP gives better recognition accuracy, and performs faster during testing, but takes more time to train, which is not a problem when considering offline classifications. The modified SFAM, like its predecessors, SFAM and fuzzy ARTMAP (FAM), is still capable of incremental learning and flexible network configuration, unlike MLP. The shortcomings of some comparative studies using MLP and FAM are also discussed. The discussion is useful, and most of the points are worthy of consideration. It should be noted, however, by the general reader that some points should not be taken at face value; some of the mentioned comparative studies used NNs for tasks other than classification applications, like function approximation. In general, the paper is well written, and the informal approach makes it easier to follow than the original FAM or SFAM papers. Anyone interested in using the FAM family of classifiers should definitely read this paper. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

Neural Processing Letters Volume 17, Issue 3

June 2003

99 pages

ISSN:1370-4621

Issue’s Table of Contents

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 June 2003

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Bakirov RFay DGabrys B(2021)Automated adaptation strategies for stream learningMachine Language10.1007/s10994-021-05992-x110:6(1429-1462)Online publication date: 1-Jun-2021
https://dl.acm.org/doi/10.1007/s10994-021-05992-x
Brito da Silva LElnabarawy IWunsch D(2019)A survey of adaptive resonance theory neural network models for engineering applicationsNeural Networks10.1016/j.neunet.2019.09.012120:C(167-203)Online publication date: 1-Dec-2019
https://dl.acm.org/doi/10.1016/j.neunet.2019.09.012
de Oliveira MInman D(2017)Performance analysis of simplified Fuzzy ARTMAP and Probabilistic Neural Networks for identifying structural damage growthApplied Soft Computing10.1016/j.asoc.2016.12.02052:C(53-63)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.1016/j.asoc.2016.12.020
Zhang YJi HZhang W(2014)TPPFAMNeurocomputing10.1016/j.neucom.2013.07.042124(63-71)Online publication date: 1-Jan-2014
https://dl.acm.org/doi/10.1016/j.neucom.2013.07.042
Cano-Izquierdo JPinzolas MGómez-Sánchez EAraúzo-Bravo MIbarrola J(2013)How well Fuzzy ARTMAP approximates functions?Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.5555/2596241.259624925:2(335-350)Online publication date: 1-Mar-2013
https://dl.acm.org/doi/10.5555/2596241.2596249
Sasu LAndonie R(2013)Bayesian ARTMAP for regressionNeural Networks10.1016/j.neunet.2013.04.00646(23-31)Online publication date: 1-Oct-2013
https://dl.acm.org/doi/10.1016/j.neunet.2013.04.006
BerghöFer ESchulze DRauch CTscherepanow MKöHler TWachsmuth S(2013)ART-based fusion of multi-modal perception for robotsNeurocomputing10.1016/j.neucom.2012.08.035107(11-22)Online publication date: 1-May-2013
https://dl.acm.org/doi/10.1016/j.neucom.2012.08.035
Marungsri BBoonpoke S(2011)Applications of simplified fuzzy ARTMAP to partial discharge classification and pattern recognitionWSEAS TRANSACTIONS on SYSTEMS10.5555/2064867.206486910:3(69-80)Online publication date: 1-Mar-2011
https://dl.acm.org/doi/10.5555/2064867.2064869
Andonie RFabry-Asztalos LAbdul-Wahid CAbdul-Wahid SBarker GMagill L(2011)Fuzzy ARTMAP Prediction of Biological Activities for Potential HIV-1 Protease Inhibitors Using a Small Molecular Data SetIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2009.508:1(80-93)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.1109/TCBB.2009.50
Su MHuang DChen JLu WTsai LLin J(2011)Mapping multi-spectral remote sensing images using rule extraction approachExpert Systems with Applications: An International Journal10.1016/j.eswa.2011.04.08638:10(12917-12922)Online publication date: 15-Sep-2011
https://dl.acm.org/doi/10.1016/j.eswa.2011.04.086
Show More Cited By

Abstract

References

Cited By

Index Terms

Recommendations

Training RBF networks with selective backpropagation

Clearly defined architectures of neural networks and multilayer perceptron

Electric load forecasting using a fuzzy ART&ARTMAP neural network

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Share

Share this Publication link

Share on social media

Affiliations