Parameter-free spectral kernel learning
Authors: 
Qi Mao, Ivor W. TsangAuthors Info & Claims
Qi Mao, Ivor W. TsangAuthors Info & ClaimsPages 350 - 357
Abstract
Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a new kernel. Interestingly, the new kernel matrix can be obtained analytically with the use of spectral decomposition of graph Laplacian matrix. Hence, the proposed algorithm does not require any numerical optimization solvers. Moreover, by maximizing kernel target alignment on labeled data, we can also learn model parameters automatically with a closed-form solution. For a given graph Laplacian matrix, our proposed method does not need to tune any model parameter including the tradeoff parameter in RLS and the balance parameter for unlabeled data. Extensive experiments on ten benchmark datasets show that our proposed two-stage parameter-free spectral kernel learning algorithm can obtain comparable performance with fine-tuned manifold regularization methods in transductive setting, and outperform multiple kernel learning in supervised setting.
References
[1]
F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan. Multiple kernel learning, conic duality, and the SMO algorithm. In ICML, 2004.
[2]
M. Belkin, P. Niyogi, V. Sindhwani, and P. Bartlett. Manifold regularization: A geometric framework for learning from examples. JMLR, 7:2399-2434, 2006.
[3]
C.-C. Chang and C. J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
[4]
O. Chapelle, J. Weston, and B. Schölkopf. Cluster kernels for semi-supervised learning. In NIPS, 2002.
[5]
O. Chapelle, B. Schölkopf, and A. Zien. Semi-supervised learning. MIT Press, 2006.
[6]
C. Cortes, M. Mohri, and A. Rostamizadeh. l2 regularization for learning kernels. In UAI, 2009a.
[7]
C. Cortes, M. Mohri, and A. Rostamizadeh. Learning non-linear combinations of kernels. In NIPS, 2009b.
[8]
N. Cristianini, J. Kandola, A. Elisseeff, and J. Shawe-Taylor. On kernel target alignment. JMLR, 1, 2002.
[9]
L. Duan, I. W. Tsang, D. Xu, and S. J. Maybank. Domain transfer SVM for video concept detection. In CVPR, 2009.
[10]
S. C. H. Hoi, R. Jin, and M. R. Lyu. Learning nonparametric kernel matrices from pairwise constraints. In ICML, 2007.
[11]
M. Kloft, U. Brefeld, P. Sonnenburg, S. amd Laskov, K.-R. Muller, and Zien A. Efficient and accurate lp-norm multiple kernel learning. In NIPS, 2009.
[12]
R. I. Kondor and J. Lafferty. Diffusion kernels on graphs and other discrete structures. In ICML, 2002.
[13]
J. T. Kwok and I. W. Tsang. Learning with idealized kernels. In ICML, 2003.
[14]
G. R.G. Lanckriet, N. Cristianini, P. Bartlett, L. B. Ghaoui, and M. I. Jordan. Learning the kernel matrix with semidefinite programming. JMLR, 5:27-72, 2004.
[15]
W. Liu, B. Qian, J. Cui, and J. Liu. Spectral kernel learning for semi-supervised classification. In IJCAI, 2009.
[16]
J. S. Nath, G. Dinesh, S. Raman, C. Bhattacharyya, A. Ben-Tal, and K. R. Tamakrishnan. On the algorithmics and applications of a mixed-norm based kernel learning formulation. In NIPS, 2009.
[17]
A. Rakotomamonjy, F. R. Bach, S Canu, and Y. Grandvalet. SimpleMKL. JMLR, 9:2491-2512, 2008.
[18]
A. J. Smola and R. Kondor. Kernels and regularization on graphs. In COLT, 2003.
[19]
S. Sonnenburg, G. Rätsch, C. Schäfer, and B. Schölkopf. Large scale multiple kernel learning. JMLR, 7:1531-1565, 2006.
[20]
J. Sun, X. Wu, S Yan, T.-S. Cheong, L.-F. and Chua, and J. Li. Hierarchical spatio-temporal context modeling for action recognition. In CVPR, 2009.
[21]
A. Vedaldi, V. Gulshan, M. Varma, and A. Zisserman. Multiple kernels for object detection. In ICCV, 2009.
[22]
D Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Schölkopf. Learning with local and global consistency. In NIPS, 2003.
[23]
X. Zhu, Z. Ghahramani, and J. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In ICML, 2003.
[24]
X. Zhu, J. Kandola, Z. Ghahramani, and J. Lafferty. Nonparametric transforms of graph kernels for semi-supervised learning. In NIPS, 2004.
[25]
J. Zhuang, I. W. Tsang, and S. C. H. Hoi. SimpleNPKL: Simple non-parameteric kernel learning. In ICML, 2009.
Index Terms
- Parameter-free spectral kernel learning
Index terms have been assigned to the content through auto-classification.
Recommendations
Indefinite kernel spectral learning
Introducing indefinite kernels in multi-class semi-supervised kernel spectral clustering (MSS-KSC) and kernel spectral clustering (KSC, as a special case of MSS-KSC).Feature space interpretations for indefinite KSC and indefinite MSSKSC are given.Based ...
Spectral domain-transfer learning
KDD '08: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data miningTraditional spectral classification has been proved to be effective in dealing with both labeled and unlabeled data when these data are from the same domain. In many real world applications, however, we wish to make use of the labeled data from one ...
Integrating Spectral Kernel Learning and Constraints in Semi-Supervised Classification
Recently, integrating new knowledge sources such as pairwise constraints into various classification tasks with insufficient training data has been actively studied in machine learning. In this paper, we propose a novel semi-supervised classification ...
Comments
Information & Contributors
Information
Published In
Publisher
AUAI Press
Arlington, Virginia, United States
Publication History
Published: 08 July 2010
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 30 Aug 2024