Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

SOM and Neural Gas as Graduated Nonconvexity Algorithms

  • Conference paper
Computational Science and Its Applications - ICCSA 2006 (ICCSA 2006)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3982))

Included in the following conference series:

Abstract

Convergence of the Self-Organizing Map (SOM) and Neural Gas (NG) is usually contemplated from the point of view of stochastic gradient descent. This class of algorithms is characterized by a very slow convergence rate. However we have found empirically that One-Pass realizations of SOM and NG provide good results or even improve over the slower realizations, when the performance measure is the distortion. One-Pass realizations use each data sample item only once, imposing a very fast reduction of the learning parameters that does not conform to the convergence requirements of stochastic gradient descent. That empirical evidence leads us to propose that the appropriate setting for the convergence analysis of SOM, NG and similar competitive clustering algorithms is the field of Graduated Nonconvexity algorithms. We show they can easily be put in this framework.

The work is partially supported by MEC grants DPI2003-06972 and VIMS-2003-20088-c04-04, and UPV/EHU grant UE03A07.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Ahalt, S.C., Krishnamurthy, A.K., Chen, P., Melton, D.E.: Competitive Learning Algorithms for Vector Quantization. Neural Networks 3, 277–290 (1990)

    Article  Google Scholar 

  2. Allgower, E.L., Georg, K.: Numerical Continuation Methods. An Introduction. Springer Series in Computational Mathematics, vol. 13. Springer, Heidelberg (1990)

    MATH  Google Scholar 

  3. Bodt, E., Cottrell, M., Letremy, P., Verleysen, M.: On the use of self-organizing maps to accelerate vector quantization. Neurocomputing 56, 187–203 (2004)

    Article  Google Scholar 

  4. Chan, C., Vetterli, M.: Lossy Compression of Individual Signals Based on String Matching and One-Pass Codebook Design. In: ICASSP 1995, Detroit, MI (1995)

    Google Scholar 

  5. Chinrungrueng, C., Séquin, C.: Optimal Adaptive K-Means Algorithm with Dynamic Adjustment of Learning Rate. IEEE Trans. on Neural Networks 6(1), 157–169 (1995)

    Article  Google Scholar 

  6. Fort, J.C., Letrémy, P., Cottrell, M.: Advantages and Drawbacks of the Batch Kohonen Algorithm. In: Verleysen, M. (ed.) Proc. of ESANN’2002, Brugge, pp. 223–230. Editions D Facto, Bruxelles (2002)

    Google Scholar 

  7. Fritzke, B.: The LBG-U method for vector quantization - an improvement over LBG inspired from neural networks. Neural Processing Letters 5(1), 35–45 (1997)

    Article  Google Scholar 

  8. Fukunaga, K.: Statistical Pattern Recognition. Academic Press, London (1990)

    MATH  Google Scholar 

  9. Gersho, A.: On the structure of vector quantizers. IEEE Trans. Inf. Th. 28(2), 157–166 (1982)

    Article  MathSciNet  Google Scholar 

  10. Gersho, A., Gray, R.M.: Vector Quantization and signal compression. Kluwer, Dordrecht (1992)

    MATH  Google Scholar 

  11. Gonzalez, A.I., Graña, M., d’Anjou, A., Albizuri, F.X.: A near real-time evolutive strategy for adaptive Color Quantization of image sequences. In: Joint Conference Information Sciences, vol. 1, pp. 69–72 (1997)

    Google Scholar 

  12. Gray, R.M.: Vector Quantization. IEEE ASSP 1, 4–29 (1984)

    Article  Google Scholar 

  13. Hofmann, T., Buhmann, J.M.: Competitive Learning Algorithms for Robust Vector Quantization. IEEE Trans. Signal Processing 46(6), 1665–1675 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  14. Kohonen, T.: Self-Organization and associative memory, (1988 2nd edn). Springer, Heidelberg (1984)

    MATH  Google Scholar 

  15. Kohonen, T.: The self-organising map. Neurocomputing 21, 1–6 (1998)

    Article  MATH  Google Scholar 

  16. Linde, Y., Buzo, A., Gray, R.M.: An algorithm for vector quantizer design. IEEE Trans. Comm. 28, 84–95 (1980)

    Article  Google Scholar 

  17. Martinetz, T., Berkovich, S., Schulten, K.: Neural-Gas network for vector quantization and his application to time series prediction. IEEE trans. Neural Networks 4(4), 558–569 (1993)

    Article  Google Scholar 

  18. Zhong, S., Ghosh, J.: A Unified Framework for Model-based Clustering. Journal of Machine Learning Research 4, 1001–1037 (2003)

    Article  MathSciNet  Google Scholar 

  19. Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1987)

    Google Scholar 

  20. Nielsen, M.: Graduated nonconvexity by functional focusing. IEEE Trans. Patt. Anal. Mach. Int. 19(5), 521–525 (1997)

    Article  Google Scholar 

  21. Gonzalez, A.I., Graña, M.: Controversial empirical results on batch versus one pass online algorithms. In: Proc. WSOM2005, Paris, France, September 2005, pp. 405–411 (2005)

    Google Scholar 

  22. Fort, J.C., Pagès, G.: About the Kohonen algorithm: strong or weak Self-organization? Neural Networks 9(5), 773–785 (1996)

    Article  Google Scholar 

  23. Gualtieri, J.A., Chettri, S.: Support vector machines for classification of hyperspectral data. In: Proc. Geosci. Rem. Sens. Symp., 2000, IGARSS, vol. 2, pp. 813–815 (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Grupo de Inteligencia Computacional, Facultad de Informática, UPV/EHU, Apdo. 649, 20080, San Sebastián, España

    Ana I. González, Alicia D’Anjou, M. Teresa García-Sebastian & Manuel Graña

Authors
  1. Ana I. González
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alicia D’Anjou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Teresa García-Sebastian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manuel Graña
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Calgary, 2500 University Drive N.W., T2N 1N4, Calgary, AB, Canada

    Marina Gavrilova

  2. Department of Mathematics and Computer Science, University of Perugia, via Vanvitelli, 1, I-06123, Perugia, Italy

    Osvaldo Gervasi

  3. William Norris Professor, Head of the Computer Science and Engineering Department, University of Minnesota, USA

    Vipin Kumar

  4. OptimaNumerics Ltd., Cathedral House, 23-31 Waring Street, BT1 2DX, Belfast, UK

    C. J. Kenneth Tan

  5. Clayton School of IT, Monash University, 3800, Clayton, Australia

    David Taniar

  6. Department of Chemistry, University of Perugia, Via Elce di Sotto, 8, I-06123, Perugia, Italy

    Antonio Laganá

  7. School of Computing, Soongsil University, Seoul, Korea

    Youngsong Mun

  8. School of Information and Communication Engineering, Sungkyunkwan University, Korea

    Hyunseung Choo

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

González, A.I., D’Anjou, A., García-Sebastian, M.T., Graña, M. (2006). SOM and Neural Gas as Graduated Nonconvexity Algorithms. In: Gavrilova, M., et al. Computational Science and Its Applications - ICCSA 2006. ICCSA 2006. Lecture Notes in Computer Science, vol 3982. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11751595_120

Download citation

  • DOI: https://doi.org/10.1007/11751595_120

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-34075-1

  • Online ISBN: 978-3-540-34076-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation