Article

Semi-supervised learning with the graph Laplacian: the limit of infinite unlabelled data

Authors:

Nathan Srebro, and

Xueyuan ZhouAuthors Info & Claims

NIPS'09: Proceedings of the 22nd International Conference on Neural Information Processing Systems

December 2009

Pages 1330 - 1338

Published: 07 December 2009 Publication History

Abstract

We study the behavior of the popular Laplacian Regularization method for Semi-Supervised Learning at the regime of a fixed number of labeled points but a large number of unlabeled points. We show that in ℝ^d, d ≥ 2, the method is actually not well-posed, and as the number of unlabeled points increases the solution degenerates to a noninformative function. We also contrast the method with the Laplacian Eigenvector method, and discuss the "smoothness" assumptions associated with this alternate method.

References

[1]

R.A. Adams, Sobolev Spaces, Academic Press (New York), 1975.

[2]

A. Azran, The rendevous algorithm: multiclass semi-supervised learning with Markov Random Walks, ICML, 2007.

[3]

M. Belkin, P. Niyogi, Using manifold structure for partially labelled classification, NIPS, vol. 15, 2003.

[4]

M. Belkin and P. Niyogi, Convergence of Laplacian Eigenmaps, NIPS 19, 2007.

[5]

M. Belkin, P. Niyogi and S. Sindhwani, Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples, JMLR, 7:2399-2434, 2006.

[6]

Y. Bengio, O. Delalleau, N. Le Roux, label propagation and quadratic criterion, in Semi-Supervised Learning, Chapelle, Scholkopf and Zien, editors, MIT Press, 2006.

[7]

O. Bosquet, O. Chapelle, M. Hein, Measure Based Regularization, NIPS, vol. 16, 2004.

[8]

M. Hein, Uniform convergence of adaptive graph-based regularization, COLT, 2006.

[9]

J. Lafferty, L. Wasserman, Statistical Analysis of Semi-Supervised Regression, NIPS, vol. 20, 2008.

[10]

U. von Luxburg, M. Belkin and O. Bousquet, Consistency of spectral clustering, Annals of Statistics, vol. 36(2), 2008.

[11]

M. Meila, J. Shi. A random walks view of spectral segmentation, AI and Statistics, 2001.

[12]

B. Nadler, S. Lafon, I.G. Kevrekidis, R.R. Coifman, Diffusion maps, spectral clustering and eigenfunctions of Fokker-Planck operators, NIPS, vol. 18, 2006.

[13]

B. Schölkopf, A. Smola, Learning with Kernels, MIT Press, 2002.

[14]

D. Zhou, O. Bousquet, T. Navin Lal, J. Weston, B. Schölkopf, Learning with local and global consistency, NIPS, vol. 16, 2004.

[15]

X. Zhu, Z. Ghahramani, J. Lafferty, Semi-Supervised Learning using Gaussian fields and harmonic functions, ICML, 2003.

Cited By

Chin AChen YM. Altenburger KUgander J(2019)Decoupled Smoothing on GraphsThe World Wide Web Conference10.1145/3308558.3313748(263-272)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313748
Ibrahim RGleich D(2019)Nonlinear Diffusion for Community Detection and Semi-Supervised LearningThe World Wide Web Conference10.1145/3308558.3313483(739-750)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313483
Shi ZOsher SZhu W(2017)Weighted Nonlocal Laplacian on Interpolation from Sparse DataJournal of Scientific Computing10.1007/s10915-017-0421-z73:2-3(1164-1177)Online publication date: 1-Dec-2017
https://dl.acm.org/doi/10.1007/s10915-017-0421-z
Show More Cited By

Semi-supervised learning with the graph Laplacian: the limit of infinite unlabelled data
1. Computing methodologies

Recommendations

L1-norm Laplacian support vector machine for data reduction in semi-supervised learning
Abstract
As a semi-supervised learning method, Laplacian support vector machine (LapSVM) is popular. Unfortunately, the model generated by LapSVM has a poor sparsity. A sparse decision model has always been fascinating because it could implement data ... $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$ $_{}$
Read More
Semi-supervised learning with nuclear norm regularization

Integrating new knowledge sources into various learning tasks to improve their performance has recently become an interesting topic. In this paper we propose a novel semi-supervised learning (SSL) approach, called semi-supervised learning with nuclear ...
Read More
Neighborhood graph construction for semi-supervised learning

Semi-supervised learning (SSL) is useful when few labeled and plenty of unlabeled examples are available. This occurs in most of the cases due to labeled instances be difficult, expensive and time consuming to be obtained since human experts are ...
Read More

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

NIPS'09: Proceedings of the 22nd International Conference on Neural Information Processing Systems

December 2009

2348 pages

ISBN:9781615679119

Publisher

Curran Associates Inc.

Red Hook, NY, United States

Publication History

Published: 07 December 2009

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Chin AChen YM. Altenburger KUgander J(2019)Decoupled Smoothing on GraphsThe World Wide Web Conference10.1145/3308558.3313748(263-272)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313748
Ibrahim RGleich D(2019)Nonlinear Diffusion for Community Detection and Semi-Supervised LearningThe World Wide Web Conference10.1145/3308558.3313483(739-750)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313483
Shi ZOsher SZhu W(2017)Weighted Nonlocal Laplacian on Interpolation from Sparse DataJournal of Scientific Computing10.1007/s10915-017-0421-z73:2-3(1164-1177)Online publication date: 1-Dec-2017
https://dl.acm.org/doi/10.1007/s10915-017-0421-z
Makantasis KProtopapadakis EDoulamis AMatsatsinis N(2016)Semi-supervised vision-based maritime surveillance system using fused visual attention mapsMultimedia Tools and Applications10.1007/s11042-015-2512-x75:22(15051-15078)Online publication date: 1-Nov-2016
https://dl.acm.org/doi/10.1007/s11042-015-2512-x
Ghosh SPrügel-Bennett A(2015)Ising bandits with side informationProceedings of the 2015th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I10.1007/978-3-319-23528-8_28(448-463)Online publication date: 7-Sep-2015
https://dl.acm.org/doi/10.1007/978-3-319-23528-8_28
Weiss YFergus RTorralba A(2012)Multidimensional spectral hashingProceedings of the 12th European conference on Computer Vision - Volume Part V10.1007/978-3-642-33715-4_25(340-353)Online publication date: 7-Oct-2012
https://dl.acm.org/doi/10.1007/978-3-642-33715-4_25
Gavish MNadler BCoifman R(2010)Multiscale wavelets on trees, graphs and high dimensional dataProceedings of the 27th International Conference on International Conference on Machine Learning10.5555/3104322.3104370(367-374)Online publication date: 21-Jun-2010
https://dl.acm.org/doi/10.5555/3104322.3104370
Luxburg URadl AHein M(2010)Getting lost in spaceProceedings of the 23rd International Conference on Neural Information Processing Systems - Volume 210.5555/2997046.2997188(2622-2630)Online publication date: 6-Dec-2010
https://dl.acm.org/doi/10.5555/2997046.2997188

View Options

View options

Media

Figures

Other

Tables

View Table of Contents