Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
article

Evolutionary Design of Nearest Prototype Classifiers

Published: 01 July 2004 Publication History

Abstract

In pattern classification problems, many works have been carried out with the aim of designing good classifiers from different perspectives. These works achieve very good results in many domains. However, in general they are very dependent on some crucial parameters involved in the design. These parameters have to be found by a trial and error process or by some automatic methods, like heuristic search and genetic algorithms, that strongly decrease the performance of the method. For instance, in nearest prototype approaches, main parameters are the number of prototypes to use, the initial set, and a smoothing parameter. In this work, an evolutionary approach based on Nearest Prototype Classifier (ENPC) is introduced where no parameters are involved, thus overcoming all the problems that classical methods have in tuning and searching for the appropiate values. The algorithm is based on the evolution of a set of prototypes that can execute several operators in order to increase their quality in a local sense, and with a high classification accuracy emerging for the whole classifier. This new approach has been tested using four different classical domains, including such artificial distributions as spiral and uniform distibuted data sets, the Iris Data Set and an application domain about diabetes. In all the cases, the experiments show successfull results, not only in the classification accuracy, but also in the number and distribution of the prototypes achieved.

References

[1]
Aha, D. and K. Kibler. (1991). "Instance-Based Learning Algorithms." Machine Learning 6, pp. 37-66.
[2]
Bermejo, S. and J. Cabestany. (2000). "A Batch Learning Algorithm Vector Quantization Algorithm for Nearest Neighbour Classification." Neural Processing Letters 11, pp. 173-184.
[3]
Bezdek, J.C. and L.I. Kuncheva. (2001). "Nearest Neighbour Classifier Designs: An Experimental Study." International Journal of Intelligent Systems 16, pp. 1445-1473.
[4]
Bezdek, J.C., T.R. Rechherzer, G.S. Lim, and Y. Attikiouzel. (1998). "Multiple-Prototype Classifier Design." IEEE Transactions on Systems, Man and Cybernetics 28(1), pp. 67-79.
[5]
Blake, C.L. and C.J. Merz. (1998). "UCI Repository of Machine Learning Databases."
[6]
Burrascano, P. (1991). "Learning Vector Quantization for the Probabilistic Neural Network." IEEE Transactions on Neural Networks 2(4), pp. 458-461.
[7]
Cagnoni, S. and G. Valli. (1994). "OSLVQ: A Training Strategy for Optimum-Size Learning Vector Quantization Classifiers." In IEEE International Conference in Neural Networks, pp. 762-775.
[8]
Duda, R.O. and P.E. Hart. (1973). Pattern Classification and Scene Analysis. John Wiley And Sons.
[9]
Fernández, F. and D. Borrajo. (2002). "On Determinism Handling While Learning Reduced State Space Representations." In Proceedings of the European Conference on Artificial Intelligence (ECAI 2002), Lyon, France.
[10]
Fernández, F. and P. Isasi. (2001). "Designing Nearest Neighbour Classifiers by the Evolution of a Population of Prototypes." In Proceedings of the European Symposium on Artificial Neural Networks (ESANN'01), pp. 172- 180.
[11]
Fernández, F. and P. Isasi. (2002). "Automatic Finding of Good Classifiers Following a Biologically Inspired Metaphor." Computing and Informatics 21(3), pp. 205-220.
[12]
Frank, E. and I.H. Witten. (1998). "Generating Accurate Rule SetsWithout Global Optimization." In Proceedings of the Fifteenth International Conference on Machine Learnin.
[13]
Fritzke, B. (1994). "Growing Cell Structures--A Self-Organizing Network for Unsupervised and Supervised Learning." Neural Networks 7(9), pp. 1441-1460.
[14]
Gersho, A. and R.M. Gray. (1992). Vector Quantization and Signal Compression. Kluwer Academic Publishers.
[15]
Hart, P.E. (1968). "The Condensed Nearest Neighbour Rule." IEEE Transactions on Information Theory.
[16]
John, G.H. and P. Langley. (1995). "Estimating Continuous Distributions in Bayesian Classifiers." In Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338-345.
[17]
Kohonen, T. (1984). Self-Organization and Associative Memory, 3rd ed. Berlin, Heidelberg: Springer, 1989.
[18]
Kuncheva, L.I. and J.C. Bezdek. (1998). "Nearest Prototype Classification: Clustering, Genetic Algorithms, or Random Search?" IEEE Transactions on Systems, Man and Cybernetics 28(1), pp. 160-164.
[19]
Linde, Y., A. Buzo, and R.M. Gray. (1980). "An Algorithm for Vector Quantizer Design." In IEEE Transactions on Communications, Vol. 1, Com-28, No. 1, pp. 84-95.
[20]
Lloyd, S.P. (1982). "Least Squares Quantization in PCM." In IEEE Transactions on Information Theory, pp. 127-135.
[21]
Mao, K.Z., K.-C. Tan, and W. Ser. (2000). "Probabilistic Neural-Network Structure Determination for Pattern Classification." IEEE Transactions on Neural Networks 11(4), pp. 1009-1016.
[22]
Merelo, J.J., A. Prieto, and F. Morán. (1998). "Optimization of Classifiers using Genetic Algorithms." In Honavar, P. (ed.), Advances in Evolutionary Synthesis of Neural Systems. MIT Press.
[23]
Pal, N.R., J.C. Bezdek, and E.C.K. Tsao. (1993). "Generalized Clustering Networks and Kohonen's Self-Organizing Scheme." IEEE Transactions on Neural Networks 4(4).
[24]
Patanè, G. and M. Russo. (2001). "The Enhanced LBG Algorithm." Neural Networks 14, pp. 1219-1237.
[25]
Pérez, J.C. and E. Vidal. (1993). "Constructive Design of LVQ and DSM Classifiers." In Mira, J., Cabestany, J., and Prieto, A. (eds.), New Trends in Neural Computation, Vol. 686 of Lecture Notes in Computer Science, Springer Verlag.
[26]
Quinlan, J.R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann.
[27]
Ritter, G.L., H.B. Woodruff, S.R. Lowri, and T.L. Isenhour. (1975). "An Algorithm for a Selective Nearest Neighbour Decision Rule." IEEE Transactions on Information Theory 21(6), pp. 665-669.
[28]
Russo, M. and G. Patanè. (2000). "ELBG Implementation." International Journal of Knowledge Based Intelligent Engineering Systems 2(4), pp. 94-109.
[29]
Specht, D.F. (1990). "Probabilistic Neural Networks." Neural Networks 3(1), pp. 109-118.
[30]
Wilson, D.R. and T.R. Martinez. (2000). "Reduction Techniques for Instance Based Learning Algorithms." Machine Learning 38, pp. 257-286.
[31]
Witten, I.H. and E. Frank. (2000). Data Mining. Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann.
[32]
Zhao, Q. and T. Higuchi. (1996). "Evolutionary Learning of Nearest Neighbour MLP." IEEE Transactions on Neural Networks 7(3), pp. 762-767.

Cited By

View all
  • (2021)Domain adaptation via incremental confidence samples into classificationInternational Journal of Intelligent Systems10.1002/int.2262937:1(365-385)Online publication date: 23-Nov-2021
  • (2017)An iterative genetic programming approach to prototype generationGenetic Programming and Evolvable Machines10.1007/s10710-016-9279-318:2(123-147)Online publication date: 1-Jun-2017
  • (2017)MOPGPattern Analysis & Applications10.1007/s10044-015-0454-620:1(33-47)Online publication date: 1-Feb-2017
  • Show More Cited By

Comments

Information & Contributors

Information

Published In

cover image Journal of Heuristics
Journal of Heuristics  Volume 10, Issue 4
July 2004
65 pages

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 July 2004

Author Tags

  1. classifier design
  2. evolutionary learning
  3. nearest prototype classifiers

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 23 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2021)Domain adaptation via incremental confidence samples into classificationInternational Journal of Intelligent Systems10.1002/int.2262937:1(365-385)Online publication date: 23-Nov-2021
  • (2017)An iterative genetic programming approach to prototype generationGenetic Programming and Evolvable Machines10.1007/s10710-016-9279-318:2(123-147)Online publication date: 1-Jun-2017
  • (2017)MOPGPattern Analysis & Applications10.1007/s10044-015-0454-620:1(33-47)Online publication date: 1-Feb-2017
  • (2017)Prototype generation on structural data using dissimilarity space representationNeural Computing and Applications10.1007/s00521-016-2278-828:9(2415-2424)Online publication date: 1-Sep-2017
  • (2017)Selecting promising classes from generated data for an efficient multi-class nearest neighbor classificationSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-016-2176-021:20(6183-6189)Online publication date: 1-Oct-2017
  • (2016)PGGPApplied Soft Computing10.1016/j.asoc.2015.12.01540:C(569-580)Online publication date: 1-Mar-2016
  • (2014)Simultaneous generation of prototypes and features through genetic programmingProceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation10.1145/2576768.2598356(517-524)Online publication date: 12-Jul-2014
  • (2012)A prototype-based method for classification with time constraintsPattern Analysis & Applications10.5555/2736758.273681315:3(261-277)Online publication date: 1-Aug-2012
  • (2012)A method for building prototypes in the nearest prototype approach based on similarity relations for problems of function approximationProceedings of the 11th Mexican international conference on Advances in Artificial Intelligence - Volume Part I10.1007/978-3-642-37807-2_4(39-50)Online publication date: 27-Oct-2012
  • (2010)A Survey on Evolutionary Instance Selection and GenerationInternational Journal of Applied Metaheuristic Computing10.4018/jamc.20101026041:1(60-92)Online publication date: 1-Jan-2010
  • Show More Cited By

View Options

View options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media