Article

Local distance preservation in the GP-LVM through back constraints

Authors:

Neil D. Lawrence,

Joaquin Quiñonero-CandelaAuthors Info & Claims

ICML '06: Proceedings of the 23rd international conference on Machine learning

Pages 513 - 520

https://doi.org/10.1145/1143844.1143909

Published: 25 June 2006 Publication History

Abstract

The Gaussian process latent variable model (GP-LVM) is a generative approach to nonlinear low dimensional embedding, that provides a smooth probabilistic mapping from latent to data space. It is also a non-linear generalization of probabilistic PCA (PPCA) (Tipping & Bishop, 1999). While most approaches to non-linear dimensionality methods focus on preserving local distances in data space, the GP-LVM focusses on exactly the opposite. Being a smooth mapping from latent to data space, it focusses on keeping things apart in latent space that are far apart in data space. In this paper we first provide an overview of dimensionality reduction techniques, placing the emphasis on the kind of distance relation preserved. We then show how the GP-LVM can be generalized, through back constraints, to additionally preserve local distances. We give illustrative experiments on common data sets.

References

[1]

Bilmes, J., Malkin, J., Li, X., Harada, S., Kilanski, K., Kirchhoff, K., Wright, R., Subramanya, A., Landay, J., Dowden, P., & Chizeck, H. (2006). The vocal joystick. Proceedings of the IEEE Conference on Acoustics, Speech and Signal Processing. IEEE. To appear.

[2]

Bishop, C. M., Svensén, M., & Williams, C. K. I. (1998). GTM: the Generative Topographic Mapping. Neural Computation, 10, 215--234.

Digital Library

[3]

Grochow, K., Martin, S. L., Hertzmann, A., & Popovic, Z. (2004). Style-based inverse kinematics. ACM Trans. on Graphics (SIGGRAPH 2004).

Digital Library

[4]

Hinton, G., & Roweis, S. (2003). Stochastic neighbor embedding. Advances in Neural Information Processing Systems 15. Cambridge, MA: MIT Press.

[5]

Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995). The wake-sleep algorithm for unsupervised neural networks. Science, 268, 1158--1161.

[6]

Jolliffe, I. T. (1986). Principal component analysis. New York: Springer-Verlag.

[7]

Lawrence, N. D. (2004). Gaussian process models for visualisation of high dimensional data. Advances in Neural Information Processing Systems (pp. 329--336). Cambridge, MA: MIT Press.

[8]

Lawrence, N. D. (2005). Probabilistic non-linear principal component analysis with Gaussian process latent variable models. Journal of Machine Learning Research, 6, 1783--1816.

Digital Library

[9]

Lowe, D., & Tipping, M. E. (1996). Feed-forward neural networks and topographic mappings for exploratory data analysis. Neural Computing and Applications, 4.

[10]

MacKay, D. J. C. (1995). Bayesian neural networks and density networks. Nuclear Instruments and Methods in Physics Research, A, 354, 73--80.

[11]

Mardia, K. V., Kent, J. T., & Bibby, J. M. (1979). Multivariate analysis. London: Academic Press.

[12]

Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323--2326.

[13]

Sammon, J. W. (1969). A nonlinear mapping for data structure analysis. IEEE Transactions on Computers, C-18, 401--409.

Digital Library

[14]

Schölkopf, B., Smola, A., & Müüller, K.-R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10, 1299--1319.

Digital Library

[15]

Shon, A. P., Grochow, K., Hertzmann, A., & Rao, R. P. N. (2006). Learning shared latent structure for image synthesis and robotic imitation. In (Weiss et al., 2006).

[16]

Tenenbaum, J. B., Silva, V. d., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290, 2319--2323.

[17]

Tipping, M. E., & Bishop, C. M. (1999). Probabilistic principal component analysis. Journal of the Royal Statistical Society, B, 6, 611--622.

Digital Library

[18]

Urtasun, R., Fleet, D. J., Hertzmann, A., & Fua, P. (2005). Priors for people tracking from small training sets. IEEE International Conference on Computer Vision (ICCV) (pp. 403--410). Bejing, China: IEEE Computer Society Press.

Digital Library

[19]

Wang, J. M., Fleet, D. J., & Hertzmann, A. (2006). Gaussian process dynamical models. In (Weiss et al., 2006).

Digital Library

[20]

Weinberger, K. Q., Sha, F., & Saul, L. K. (2004). Learning a kernel matrix for nonlinear dimensionality reduction. Proceedings of the International Conference in Machine Learning (pp. 839--846). San Francisco, CA: Morgan Kauffman.

Digital Library

[21]

Weiss, Y., Schölkopf, B., & Platt, J. C. (Eds.). (2006). Advances in neural information processing systems, vol. 18. Cambridge, MA: MIT Press.

[22]

Zien, A., & Quiññonero Candela, J. (2005). Large margin non-linear embedding. Proceedings of the 22nd International Conference on Machine Learning (pp. 1065--1072). San Francisco, CA: Morgan Kaufmann.

Digital Library

Cited By

Telo JDimou DMoreno P(2024)Trajectories in Hand Synergy Spaces for in-hand manipulation2024 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC)10.1109/ICARSC61747.2024.10535931(213-218)Online publication date: 2-May-2024
https://doi.org/10.1109/ICARSC61747.2024.10535931
De Boi IEk CPenne R(2023)Surface Approximation by Means of Gaussian Process Latent Variable Models and Line Element GeometryMathematics10.3390/math1102038011:2(380)Online publication date: 11-Jan-2023
https://doi.org/10.3390/math11020380
Wang LWei HZheng YDong JZhong G(2023)Deep Error-Correcting Output CodesAlgorithms10.3390/a1612055516:12(555)Online publication date: 4-Dec-2023
https://doi.org/10.3390/a16120555
Show More Cited By

Index Terms

Local distance preservation in the GP-LVM through back constraints
1. Mathematics of computing
  1. Probability and statistics

Recommendations

Local linear Laplacian eigenmaps

We have derived local linear Laplacian eigenmaps (LLLE) a manifold learning algorithm.The LLLE finds multiple local linear structures.The LLLE used P artificial neighbors to construct the adjacency graph.The experimental results indicate that it can ...
Discriminative information preservation for face recognition

It is usually difficult to find the optimal low dimensional subspace for face recognition. Patch alignment framework (PAF) is an important systematic framework that can be applied to understand the common thought and essential differences of a numerous ...
Local Tangent Space Discriminant Analysis

We propose a novel supervised dimensionality reduction method named local tangent space discriminant analysis (TSD) which is capable of utilizing the geometrical information from tangent spaces. The proposed method aims to seek an embedding space where ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '06: Proceedings of the 23rd international conference on Machine learning

June 2006

1154 pages

ISBN:1595933832

DOI:10.1145/1143844

Program Chairs:
William Cohen,
Andrew Moore

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

ICML '06 Paper Acceptance Rate 140 of 548 submissions, 26%;

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

136
Total Citations
View Citations
687
Total Downloads

Downloads (Last 12 months)38
Downloads (Last 6 weeks)6

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Telo JDimou DMoreno P(2024)Trajectories in Hand Synergy Spaces for in-hand manipulation2024 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC)10.1109/ICARSC61747.2024.10535931(213-218)Online publication date: 2-May-2024
https://doi.org/10.1109/ICARSC61747.2024.10535931
De Boi IEk CPenne R(2023)Surface Approximation by Means of Gaussian Process Latent Variable Models and Line Element GeometryMathematics10.3390/math1102038011:2(380)Online publication date: 11-Jan-2023
https://doi.org/10.3390/math11020380
Wang LWei HZheng YDong JZhong G(2023)Deep Error-Correcting Output CodesAlgorithms10.3390/a1612055516:12(555)Online publication date: 4-Dec-2023
https://doi.org/10.3390/a16120555
Saito NMaeda KOgawa TAsamizu SHaseyama M(2023)Visual Emotion Recognition Through Multimodal Cyclic-Label Dequantized Gaussian Process Latent Variable ModelJournal of Robotics and Mechatronics10.20965/jrm.2023.p132135:5(1321-1330)Online publication date: 20-Oct-2023
https://doi.org/10.20965/jrm.2023.p1321
JIN XGUO J(2023)Regressive Gaussian Process Latent Variable Model for Few-Frame Human Motion PredictionIEICE Transactions on Information and Systems10.1587/transinf.2023PCP0001E106.D:10(1621-1626)Online publication date: 1-Oct-2023
https://doi.org/10.1587/transinf.2023PCP0001
Yoshikawa YIwata T(2023)Gaussian Process Regression With Interpretable Sample-Wise Feature WeightsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2021.313123434:9(5789-5803)Online publication date: Sep-2023
https://doi.org/10.1109/TNNLS.2021.3131234
Amadio FDelgado-Guerrero JColomé ATorras C(2023)Controlled Gaussian process dynamical models with application to robotic cloth manipulationInternational Journal of Dynamics and Control10.1007/s40435-023-01205-611:6(3209-3219)Online publication date: 16-May-2023
https://doi.org/10.1007/s40435-023-01205-6
Dimou DSantos-Victor JMoreno P(2023)Robotic hand synergies for in-hand regrasping driven by object informationAutonomous Robots10.1007/s10514-023-10101-zOnline publication date: 11-Apr-2023
https://doi.org/10.1007/s10514-023-10101-z
Papadopoulos GSilavong FMoran S(2023)A Baseline Generative Probabilistic Model for Weakly Supervised LearningMachine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track10.1007/978-3-031-43427-3_3(36-50)Online publication date: 17-Sep-2023
https://doi.org/10.1007/978-3-031-43427-3_3
Gunawardena RSarrigiannis PBlackburn DHe F(2022)Kernel-based Nonlinear Manifold Learning for EEG Functional Connectivity Analysis with Application to Alzheimer's Disease2022 IEEE Signal Processing in Medicine and Biology Symposium (SPMB)10.1109/SPMB55497.2022.10014969(1-5)Online publication date: 3-Dec-2022
https://doi.org/10.1109/SPMB55497.2022.10014969
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents