research-article

Localized multiple kernel learning

Authors:

Ethem AlpaydinAuthors Info & Claims

ICML '08: Proceedings of the 25th international conference on Machine learning

Pages 352 - 359

https://doi.org/10.1145/1390156.1390201

Published: 05 July 2008 Publication History

Abstract

Recently, instead of selecting a single kernel, multiple kernel learning (MKL) has been proposed which uses a convex combination of kernels, where the weight of each kernel is optimized during training. However, MKL assigns the same weight to a kernel over the whole input space. In this paper, we develop a localized multiple kernel learning (LMKL) algorithm using a gating model for selecting the appropriate kernel function locally. The localizing gating model and the kernel-based classifier are coupled and their optimization is done in a joint manner. Empirical results on ten benchmark and two bioinformatics data sets validate the applicability of our approach. LMKL achieves statistically similar accuracy results compared with MKL by storing fewer support vectors. LMKL can also combine multiple copies of the same kernel function localized in different parts. For example, LMKL with multiple linear kernels gives better accuracy results than using a single linear kernel on bioinformatics data sets.

References

[1]

Alpaydin, E. (1999). Combined 5 x 2 cv F test for comparing supervised classification learning algorithms. Neural Computation, 11, 1885--1892.

Digital Library

[2]

Amari, S., & Wu, S. (1998). Improving support vector machine classifiers by modifying kernel functions. Neural Networks, 12, 783--789.

Digital Library

[3]

Bach, F. R., Lanckriet, G. R. G., & Jordan, M. I. (2004). Multiple kernel learning, conic duality, and the SMO algorithm. Proceedings of the 21st International Conference on Machine Learning (pp. 41--48).

Digital Library

[4]

Collobert, R., Bengio, S., & Bengio, Y. (2001). A parallel mixture of SVMs for very large scale problems. Advances in Neural Information Processing Systems (NIPS) (pp. 633--640).

Digital Library

[5]

Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural Computation, 3, 79--87.

[6]

Lanckriet, G. R. G., Bie, T. D., Cristianini, N., Jordan, M. I., & Noble, W. S. (2004a). A statistical framework for genomic data fusion. Bioinformatics, 20, 2626--2635.

Digital Library

[7]

Lanckriet, G. R. G., Cristianini, N., Bartlett, P., Ghaoui, L. E., & Jordan, M. I. (2004b). Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research, 5, 27--72.

Digital Library

[8]

Lee, W., Verzakov, S., & Duin, R. P. W. (2007). Kernel combination versus classifier combination. Proceedings of the 7th International Workshop on Multiple Classifier Systems (pp. 22--31).

Digital Library

[9]

Lewis, D. P., Jebara, T., & Noble, W. S. (2006). Nonstationary kernel combination. Proceedings of the 23rd International Conference on Machine Learning (pp. 553--560).

Digital Library

[10]

Moguerza, J. M., Muñoz, A., & de Diego, I. M. (2004). Improving support vector classification via the combination of multiple sources of information. Proceedings of Structural, Syntactic, and Statistical Pattern Recognition, Joint IAPR International Workshops (pp. 592--600).

[11]

Mosek (2008). The MOSEK optimization tools manual version 5.0 (revision 79). MOSEK ApS, Denmark.

[12]

Pavlidis, P., Weston, J., Cai, J., & Grundy, W. N. (2001). Gene functional classification from heterogeneous data. Proceedings of the 5th Annual International Conference on Computational Molecular Biology (pp. 242--248).

Digital Library

[13]

Pedersen, A. G., & Nielsen, H. (1997). Neural network prediction of translation initiation sites in eukaryotes: Perspectives for EST and genome analysis. Proceedings of the 5th International Conference on Intelligent Systems for Molecular Biology (pp. 226--233).

Digital Library

[14]

Rakotomamonjy, A., Bach, F., Canu, S., & Grandvalet, Y. (2007). More efficiency in multiple kernel learning. Proceedings of the 24th International Conference on Machine Learning (pp. 775--782).

Digital Library

[15]

Sonnenburg, S., Rätsch, G., Schäfer, C., & Schöölkopf, B. (2006). Large scale multiple kernel learning. Journal of Machine Learning Research, 7, 1531--1565.

Digital Library

Cited By

Briscik MTazza GVidács LDillies MDéjean S(2024)Supervised multiple kernel learning approaches for multi-omics data integrationBioData Mining10.1186/s13040-024-00406-917:1Online publication date: 23-Nov-2024
https://doi.org/10.1186/s13040-024-00406-9
Li MZhang YMa CLiu SLiu ZYin JLiu XLiao Q(2024) Regularized Simple Multiple Kernel k -Means With Kernel Average Alignment IEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.329021935:11(15910-15919)Online publication date: Nov-2024
https://doi.org/10.1109/TNNLS.2023.3290219
Ding XCui MLi YChen S(2024)A maximal accuracy and minimal difference criterion for multiple kernel learningExpert Systems with Applications10.1016/j.eswa.2024.124378254(124378)Online publication date: Nov-2024
https://doi.org/10.1016/j.eswa.2024.124378
Show More Cited By

Index Terms

Localized multiple kernel learning
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees
  2. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation
      2. Modeling methodologies

Recommendations

Localized Multiple Kernel Regression
ICPR '10: Proceedings of the 2010 20th International Conference on Pattern Recognition

Multiple kernel learning (MKL) uses a weighted combination of kernels where the weight of each kernel is optimized during training. However, MKL assigns the same weight to a kernel over the whole input space. Our main objective is the formulation of the ...
Localized algorithms for multiple kernel learning

Instead of selecting a single kernel, multiple kernel learning (MKL) uses a weighted sum of kernels where the weight of each kernel is optimized during training. Such methods assign the same weight to a kernel over the whole input space, and we discuss ...
A pre-selecting base kernel method in multiple kernel learning

The pre-defined base kernel greatly affects the performance of multiple kernel learning (MKL), but selecting the pre-defined base kernel still has no theoretical guidance. In practice, it is very difficult to select a set of appropriate base kernels ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '08: Proceedings of the 25th international conference on Machine learning

July 2008

1310 pages

ISBN:9781605582054

DOI:10.1145/1390156

General Chair:
William Cohen
Carnegie Mellon University
,
Program Chairs:
Andrew McCallum
University of Massachusetts Amherst
,
Sam Roweis
University of Toronto and Google

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Pascal
University of Helsinki
Xerox
Federation of Finnish Learned Societies
Google Inc.
NSF
Machine Learning Journal/Springer
Microsoft Research: Microsoft Research
Intel: Intel
Yahoo!
Helsinki Institute for Information Technology
IBM: IBM

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Conference

ICML '08

Sponsor:

Microsoft Research
Intel
IBM

ICML '08: The 25th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

July 5 - 9, 2008

Helsinki, Finland

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

186
Total Citations
View Citations
808
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Briscik MTazza GVidács LDillies MDéjean S(2024)Supervised multiple kernel learning approaches for multi-omics data integrationBioData Mining10.1186/s13040-024-00406-917:1Online publication date: 23-Nov-2024
https://doi.org/10.1186/s13040-024-00406-9
Li MZhang YMa CLiu SLiu ZYin JLiu XLiao Q(2024) Regularized Simple Multiple Kernel k -Means With Kernel Average Alignment IEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.329021935:11(15910-15919)Online publication date: Nov-2024
https://doi.org/10.1109/TNNLS.2023.3290219
Ding XCui MLi YChen S(2024)A maximal accuracy and minimal difference criterion for multiple kernel learningExpert Systems with Applications10.1016/j.eswa.2024.124378254(124378)Online publication date: Nov-2024
https://doi.org/10.1016/j.eswa.2024.124378
KAMATA SMINE T(2023)Comments on Quasi-Linear Support Vector Machine for Nonlinear ClassificationIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences10.1587/transfun.2022EAL2051E106.A:11(1444-1445)Online publication date: 1-Nov-2023
https://doi.org/10.1587/transfun.2022EAL2051
Niewiadomski RBeyan CSciutti A(2023)Affect Recognition in Hand-Object Interaction Using Object-Sensed Tactile and Kinematic DataIEEE Transactions on Haptics10.1109/TOH.2022.323064316:1(112-117)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TOH.2022.3230643
Rahimzadeh Arashloo S(2023)Unknown Face Presentation Attack Detection via Localized Learning of Multiple KernelsIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.324084118(1421-1432)Online publication date: 2023
https://doi.org/10.1109/TIFS.2023.3240841
Chen ZXu TWu XWang RKittler J(2023)Hybrid Riemannian Graph-Embedding Metric Learning for Image Set ClassificationIEEE Transactions on Big Data10.1109/TBDATA.2021.31130849:1(75-92)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TBDATA.2021.3113084
Viazzi ARios ABlandón JGonzález J(2023)A centered kernel alignment-based strategy for pest evolution tracing: Myiopsitta monachus case2023 IEEE 13th International Conference on Pattern Recognition Systems (ICPRS)10.1109/ICPRS58416.2023.10179040(1-5)Online publication date: 4-Jul-2023
https://doi.org/10.1109/ICPRS58416.2023.10179040
Hashimoto NTakagi YMasuda HMiyoshi HKohno KNagaishi MSato KTakeuchi MFuruta TKawamoto KYamada KMoritsubo MInoue KShimasaki YOgura YImamoto TMishina TTanaka KKawaguchi YNakamura SOhshima KHontani HTakeuchi I(2023)Case-based similar image retrieval for weakly annotated large histopathological images of malignant lymphoma using deep metric learningMedical Image Analysis10.1016/j.media.2023.10275285(102752)Online publication date: Apr-2023
https://doi.org/10.1016/j.media.2023.102752
Rathod BVanzara RPandya D(2023)A recent survey on perceived group sentiment analysisJournal of Visual Communication and Image Representation10.1016/j.jvcir.2023.10398897(103988)Online publication date: Dec-2023
https://doi.org/10.1016/j.jvcir.2023.103988
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents