Article

Independent subspace analysis using geodesic spanning trees

Authors:

Barnabás Póczos,

András LõrinczAuthors Info & Claims

ICML '05: Proceedings of the 22nd international conference on Machine learning

Pages 673 - 680

https://doi.org/10.1145/1102351.1102436

Published: 07 August 2005 Publication History

Abstract

A novel algorithm for performing Independent Subspace Analysis, the estimation of hidden independent subspaces is introduced. This task is a generalization of Independent Component Analysis. The algorithm works by estimating the multi-dimensional differential entropy. The estimation utilizes minimal geodesic spanning trees matched to the sample points. Numerical studies include (i) illustrative examples, (ii) a generalization of the cocktail-party problem to songs played by bands, and (iii) an example on mixed independent subspaces, where subspaces have dependent sources, which are pairwise independent.

References

[1]

Akaho, S., Kiuchi, Y., & Umeyama, S. (1999). MICA: Multimodal independent component analysis. Proc. IJCNN (pp. 927--932).

[2]

Amari, S., Cichocki, A., & Yang, H. (1996). A new learning algorithm for blind source separation. NIPS (pp. 757--763).

[3]

Bach, F. R., & Jordan, M. I. (2002). Kernel independent component analysis. JMLR, 3, 1--48.

Digital Library

[4]

Bach, F. R., & Jordan, M. I. (2003). Finding clusters in independent component analysis. Fourth Int. Symp. on ICA and BSS (pp. 891--896).

[5]

Banks, D., Lavine, M., & Newton, H. J. (1992). The minimal spanning tree for nonparametric regression and structure discovery. Comput. Sci. and Stat., 24, 370--374.

[6]

Bell, A. J., & Sejnowski, T. J. (1995). An information maximisation approach to blind separation and blind deconvolution. Neural Comp., 7, 1129--1159.

Digital Library

[7]

Bell, A. J., & Sejnowski, T. J. (1997). The 'independent components' of natural scenes are edge filters. Vision Research, 37, 3327--3338.

[8]

Cardoso. J. (1998). Multidimensional independent component analysis. Proc. ICA SSP'98, Seattle, WA.

[9]

Comon, P. (1994). Independent component analysis, a new concept? Signal Proc., 36, 287--314.

Digital Library

[10]

Costa, J. A., & Hero, A. O. (2004). Geodesic entropic graphs for dimension and entropy estimation in manifold learning. IEEE Trans. on Signal Proc., 52, 2210--2221.

Digital Library

[11]

Gretton, A., Herbrich, R., & Smola, A. (2003). The kernel mutual information. Proc. ICASSP.

[12]

Hero, A., & Michel, O. (1998). Robust entropy estimation strategies based on edge weighted random graphs. Proc. SPIE98 (pp. 250--261). San Diego, CA.

[13]

Hild, K. E., Erdogmus, D., & Prííncipe, J. (2001). Blind source separation using Renyi's mutual information. IEEE Signal Proc. Letters, 8, 174--176.

[14]

Hyvärinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: John Wiley.

[15]

Hyvärinen, A. (1999). Sparse code shrinkage: Denoising of nongaussian data by maximum likelihood estimation. Neural Comp., 11, 1739--1768.

Digital Library

[16]

Hyvärinen, A., & Hoyer, P. (2000). Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces. Neural Comp., 12, 1705--1720.

Digital Library

[17]

Jutten, C., & Herault, J. (1991). Blind separation of sources: An adaptive algorithm based on neuromimetic architecture. Signal Proc., 24, 1--10.

Digital Library

[18]

Kiviluoto, K., & Oja, E. (1998). Independent component analysis for parallel financial time series. Proc. ICONIP'98 (pp. 895--898).

[19]

Learned-Miller, E. G., & Fisher, J. W. (2003). ICA using spacings estimates of entropy. JMLR, 4, 1271--1295.

Digital Library

[20]

Makeig, S., Bell, A. J., Jung, T. P., & Sejnowski, T. J. (1996). Independent component analysis of electroencephalographic data. NIPS (pp. 145--151).

[21]

Tenenbaum, J. B., de Silva, V., & Langford., J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290, 626--634.

[22]

Vigário, R., Jousmaki, V., Hamalainen, M., Hari, R., & Oja, E. (1998). Independent component analysis for identification of artifacts in magnetoencephalo-graphic recordings. NIPS (pp. 229--235).

Digital Library

[23]

Vollgraf, R., & Obermayer, K. (2001). Multidimensional ICA to separate correlated sources. NIPS (pp. 993--1000).

[24]

Yukich, J. E. (1998). Probability theory of classical euclidean optimization problems, vol. 1675 of Lecture Notes in Math. Springer-Verlag, Berlin.

Cited By

Du KSwamy MDu KSwamy M(2019)Independent Component AnalysisNeural Networks and Statistical Learning10.1007/978-1-4471-7452-3_15(447-482)Online publication date: 13-Sep-2019
https://doi.org/10.1007/978-1-4471-7452-3_15
Gupta AYilmaz A(2017)Subspace Projection Methods for Large Scale Image Data Analysis2017 IEEE Third International Conference on Multimedia Big Data (BigMM)10.1109/BigMM.2017.75(260-267)Online publication date: Apr-2017
https://doi.org/10.1109/BigMM.2017.75
Singh SPóczos B(2016)Finite-sample analysis of fixed-k nearest neighbor density functional estimatorsProceedings of the 30th International Conference on Neural Information Processing Systems10.5555/3157096.3157233(1225-1233)Online publication date: 5-Dec-2016
https://dl.acm.org/doi/10.5555/3157096.3157233
Show More Cited By

Independent subspace analysis using geodesic spanning trees
1. Mathematics of computing
  1. Discrete mathematics
    1. Graph theory
2. Theory of computation

Recommendations

Riemannian optimization method on the flag manifold for independent subspace analysis
ICA'06: Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation

Recent authors have investigated the use of manifolds and Lie group methods for independent component analysis (ICA), including the Stiefel and the Grassmann manifolds and the orthogonal group O(n). In this paper we introduce a new class of manifold, ...
(t, s)-Completely Independent Spanning Trees
WALCOM: Algorithms and Computation
Abstract
In this paper we first define (t, s)-completely independent spanning trees, which is a generalization of completely independent spanning trees. A set of t spanning trees of a graph is (t, s)-completely independent if, for any pair of vertices u ...
Discriminant Subspace Analysis: A Fukunaga-Koontz Approach

The Fisher Linear Discriminant (FLD) is commonly used in pattern recognition. It finds a linear subspace that maximally separates class patterns according to the Fisher Criterion. Several methods of computing the FLD have been proposed in the literature,...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '05: Proceedings of the 22nd international conference on Machine learning

August 2005

1113 pages

ISBN:1595931805

DOI:10.1145/1102351

General Chair:
Saso Dzeroski
Jozef Stefan Institute, Slovenia
,
Program Chairs:
Luc De Raedt,
Stefan Wrobel

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
272
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Du KSwamy MDu KSwamy M(2019)Independent Component AnalysisNeural Networks and Statistical Learning10.1007/978-1-4471-7452-3_15(447-482)Online publication date: 13-Sep-2019
https://doi.org/10.1007/978-1-4471-7452-3_15
Gupta AYilmaz A(2017)Subspace Projection Methods for Large Scale Image Data Analysis2017 IEEE Third International Conference on Multimedia Big Data (BigMM)10.1109/BigMM.2017.75(260-267)Online publication date: Apr-2017
https://doi.org/10.1109/BigMM.2017.75
Singh SPóczos B(2016)Finite-sample analysis of fixed-k nearest neighbor density functional estimatorsProceedings of the 30th International Conference on Neural Information Processing Systems10.5555/3157096.3157233(1225-1233)Online publication date: 5-Dec-2016
https://dl.acm.org/doi/10.5555/3157096.3157233
Ball KBigdely-Shamlo NMullen TRobbins K(2016)PWC-ICAComputational Intelligence and Neuroscience10.1155/2016/97548132016(20)Online publication date: 1-Jun-2016
https://dl.acm.org/doi/10.1155/2016/9754813
Elhoseiny MElgammal A(2015)Generalized Twin Gaussian processes using Sharma---Mittal divergenceMachine Language10.1007/s10994-015-5497-9100:2-3(399-424)Online publication date: 1-Sep-2015
https://dl.acm.org/doi/10.1007/s10994-015-5497-9
Almeida MVigario RBioucas-Dias J(2014)Separation of Synchronous Sources Through Phase Locked Matrix FactorizationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2013.229779125:10(1894-1908)Online publication date: Oct-2014
https://doi.org/10.1109/TNNLS.2013.2297791
Szabó ZPóCzos BLrincz A(2012)Separation theorem for independent subspace analysis and its consequencesPattern Recognition10.1016/j.patcog.2011.09.00745:4(1782-1791)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.1016/j.patcog.2011.09.007
Pál DPóczos BSzepesvári C(2010)Estimation of Rényi entropy and mutual information based on generalized nearest-neighbor graphsProceedings of the 23rd International Conference on Neural Information Processing Systems - Volume 210.5555/2997046.2997102(1849-1857)Online publication date: 6-Dec-2010
https://dl.acm.org/doi/10.5555/2997046.2997102
Szabó ZPóczos BLőrincz A(2010)Auto-regressive independent process analysis without combinatorial effortsPattern Analysis & Applications10.5555/2736754.273679013:1(1-13)Online publication date: 1-Feb-2010
https://dl.acm.org/doi/10.5555/2736754.2736790
Szabó ZLörincz A(2009)Controlled complete ARMA independent process analysisProceedings of the 2009 international joint conference on Neural Networks10.5555/1704175.1704395(1521-1528)Online publication date: 14-Jun-2009
https://dl.acm.org/doi/10.5555/1704175.1704395
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents