Article

Hierarchic Bayesian models for kernel learning

Authors:

Simon RogersAuthors Info & Claims

ICML '05: Proceedings of the 22nd international conference on Machine learning

Pages 241 - 248

https://doi.org/10.1145/1102351.1102382

Published: 07 August 2005 Publication History

Abstract

The integration of diverse forms of informative data by learning an optimal combination of base kernels in classification or regression problems can provide enhanced performance when compared to that obtained from any single data source. We present a Bayesian hierarchical model which enables kernel learning and present effective variational Bayes estimators for regression and classification. Illustrative experiments demonstrate the utility of the proposed method. Matlab code replicating results reported is available at http://www.dcs.gla.ac.uk/~srogers/kernel_comb.html.

References

[1]

Andrews, D., & Mallows, C. (1974). Scale mixtures of Normal distributions. Journal of the Royal Statistical Society, Series B, 36, 99--102.]]

[2]

Bach, F., Lanckriet, G., & Jordan, M. I. (2004). Multiple kernel learning, conic duality and the SMO algorithm. Proceedings of the Twenty-First International Conference on Machine Learning.]]

Digital Library

[3]

Bach, F. R., Thibaux, R., & Jordan, M. I. (2005). Computing regularization paths for learning multiple kernels. In L. K. Saul. Y. Weiss and L. Bottou (Eds.), Advances in neural information processing systems 17. Cambridge. MA: MIT Press.]]

[4]

Beal, M. (2003). Variational algorithms for approximate bayesian inference. Doctoral dissertation, University College London.]]

[5]

Bishop, C., & Tipping, M. (2000). Variational relevance vector machines. Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (pp.46--53).]]

Digital Library

[6]

Bousquet, O., & Herrmann, D. J. L. (2003). On the complexity of learning the kernel matrix. In S. T. S. Becker and K. Obermayer (Eds.), Advances in neural information processing systems 15, 399--406. Cambridge, MA: MIT Press.]]

[7]

Crammer, K., Keshet, J., & Singer, Y. (2003). Kernel design using boosting. In S. T. S. Becker and K. Obermayer (Eds.), Advances in neural information processing systems 15, 537--544. Cambridge, MA: MIT Press.]]

[8]

Cristianini, N., Shawe-Taylor, J., Elisseeff, A., & Kandola. J. (2002). On kernel-target alignment. Advances in Neural Information Processing Systems 14. Cambridge, MA: MIT Press.]]

[9]

Fung, G., Dundar, M., Bi, J., & Rao, B. (2004). A fast iterative algorithm for Fisher discriminant using heterogeneous kernels. Proceedings of the twenty-first International Conference on Machine Learning (pp. 313--320).]]

Digital Library

[10]

Gunn, S., & Kandola, J. (2002). Structural modelling with sparse kernels. Machine Learning, 48, 137--163.]]

Digital Library

[11]

Jaakkola, T. (1997). Variational methods for inference and estimation in graphical models. Doctoral dissertation, MIT.]]

Digital Library

[12]

Jordan, M., Ghahramani, Z., Jaakkola, T., & Saul, L. (1999). An introduction to variational methods for graphical models. Machine Learning, 37, 183--233.]]

Digital Library

[13]

Kolenda, T., Hansen, L., Larsen, J., & Winther, O. (2002). Independent component analysis for understanding multimedia content. Proceedings of IEEE Workshop on Neural Networks for Signal Processing XII (pp. 757--766).]]

[14]

Lafferty, J., & Lebanon, G. (2005). Diffusion kernels on statistical manifolds. Journal of Machine Learning Research, 6, 129--163.]]

Digital Library

[15]

Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L. E., & Jordan, M. (2004). Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research, 5, 27--72.]]

Digital Library

[16]

Lawrence, N. D., Milo, M., Niranjan, M., Rashbass, P., & Soullier, S. (2004). Reducing the variability in cDNA microarray image processing by Bayesian inference. Bioinformatics, 20, 518--526.]]

Digital Library

[17]

MacKay, D. (2003). Information theory, inference, and learning algorithms. Cambridge University Press.]]

Digital Library

[18]

Minka, T. (2001). A family of algorithms for approximate Bayesian inference. Doctoral dissertation, MIT.]]

Digital Library

[19]

Ong, C. S., Smola, A. J., & Williamson, R. C. (2003). Hyperkernels. In S. T. S. Becker and K. Obermayer (Eds.), Advances in neural information processing systems 15, 478--485. Cambridge, MA: MIT Press.]]

[20]

Shawe-Taylor, J., & Cristianini, N. (2004). Kernel methods for pattern analysis. Cambridge: Cambridge University Press.]]

Digital Library

[21]

Tipping, M. (2001). Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1, 211--244.]]

Digital Library

[22]

Tsang, I. W., & Kwok, J. T. (2004). Efficient hyperkernel learning using second-order cone programming. Proceedings of the 15th European Conference on Machine Learning (pp. 453--464).]]

Digital Library

[23]

Zhang, Z., Yeung, D.-Y., & Kwok, J. T. (2004). Bayesian inference for transductive learning of kernel matrix using the Tanner-Wong data augmentation algorithm. Proceedings of the Twenty-First International Conference on Machine Learning (pp. 935--942).]]

Digital Library

Cited By

Pyrialakos SKalogeris IPapadopoulos V(2025)An efficient hierarchical Bayesian framework for multiscale material modelingComposite Structures10.1016/j.compstruct.2024.118570351(118570)Online publication date: Jan-2025
https://doi.org/10.1016/j.compstruct.2024.118570
Liu XChen XLi JZhou XChen Y(2020)Facies Identification Based on Multikernel Relevance Vector MachineIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2020.298168758:10(7269-7282)Online publication date: Oct-2020
https://doi.org/10.1109/TGRS.2020.2981687
Oikonomou VNikolopoulos SKompatsiaris I(2019)A Bayesian Multiple Kernel Learning Algorithm for SSVEP BCI DetectionIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2018.287804823:5(1990-2001)Online publication date: Sep-2019
https://doi.org/10.1109/JBHI.2018.2878048
Show More Cited By

Recommendations

Tractable Bayesian learning of tree augmented Naive Bayes models
ICML'03: Proceedings of the Twentieth International Conference on International Conference on Machine Learning

Bayesian classifiers such as Naive Bayes or Tree Augmented Naive Bayes (TAN) have shown excellent performance given their simplicity and heavy underlying independence assumptions. In this paper we introduce a classifier taking as basis the TAN model and ...
Model-based kernel sum rule: kernel Bayesian inference with probabilistic models
Abstract
Kernel Bayesian inference is a principled approach to nonparametric inference in probabilistic graphical models, where probabilistic relationships between variables are learned from data in a nonparametric manner. Various algorithms of kernel ...
Bayesian nonparametric latent feature models

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '05: Proceedings of the 22nd international conference on Machine learning

August 2005

1113 pages

ISBN:1595931805

DOI:10.1145/1102351

General Chair:
Saso Dzeroski
Jozef Stefan Institute, Slovenia
,
Program Chairs:
Luc De Raedt,
Stefan Wrobel

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
428
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pyrialakos SKalogeris IPapadopoulos V(2025)An efficient hierarchical Bayesian framework for multiscale material modelingComposite Structures10.1016/j.compstruct.2024.118570351(118570)Online publication date: Jan-2025
https://doi.org/10.1016/j.compstruct.2024.118570
Liu XChen XLi JZhou XChen Y(2020)Facies Identification Based on Multikernel Relevance Vector MachineIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2020.298168758:10(7269-7282)Online publication date: Oct-2020
https://doi.org/10.1109/TGRS.2020.2981687
Oikonomou VNikolopoulos SKompatsiaris I(2019)A Bayesian Multiple Kernel Learning Algorithm for SSVEP BCI DetectionIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2018.287804823:5(1990-2001)Online publication date: Sep-2019
https://doi.org/10.1109/JBHI.2018.2878048
Xie JMa ZZhang GXue JChien JLin ZGuo J(2018)BALSON: BAYESIAN LEAST SQUARES OPTIMIZATION WITH NONNEGATIVE L1-NORM CONSTRAINT2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP)10.1109/MLSP.2018.8517036(1-6)Online publication date: Sep-2018
https://doi.org/10.1109/MLSP.2018.8517036
Sun WYuan CWang PYang SHu WCai Z(2017)Hierarchical Bayesian Multiple Kernel Learning Based Feature Fusion for Action RecognitionMultimodal Pattern Recognition of Social Signals in Human-Computer-Interaction10.1007/978-3-319-59259-6_8(85-97)Online publication date: 1-Jun-2017
https://doi.org/10.1007/978-3-319-59259-6_8
Sinha ADuchi J(2016)Learning kernels with random featuresProceedings of the 30th International Conference on Neural Information Processing Systems10.5555/3157096.3157242(1306-1314)Online publication date: 5-Dec-2016
https://dl.acm.org/doi/10.5555/3157096.3157242
Du CDu CLong GHe QLi Y(2016)Online Bayesian multiple kernel bipartite rankingProceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence10.5555/3020948.3020964(142-151)Online publication date: 25-Jun-2016
https://dl.acm.org/doi/10.5555/3020948.3020964
Du CDu CLong GJin XLi Y(2016)Efficient Bayesian Maximum Margin Multiple Kernel LearningEuropean Conference on Machine Learning and Knowledge Discovery in Databases - Volume 985110.1007/978-3-319-46128-1_11(165-181)Online publication date: 19-Sep-2016
https://dl.acm.org/doi/10.1007/978-3-319-46128-1_11
Madsen JJensen BLarsen JSoleymani MYang YJiang YChang S(2015)Learning Combinations of Multiple Feature Representations for Music Emotion PredictionProceedings of the 1st International Workshop on Affect & Sentiment in Multimedia10.1145/2813524.2813534(3-8)Online publication date: 30-Oct-2015
https://dl.acm.org/doi/10.1145/2813524.2813534
Qi Mao Tsang IShenghua Gao Li Wang (2015)Generalized Multiple Kernel Learning With Data-Dependent PriorsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2014.233413726:6(1134-1148)Online publication date: Jun-2015
https://doi.org/10.1109/TNNLS.2014.2334137
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents