Article

Most likely heteroscedastic Gaussian process regression

Authors:

Kristian Kersting,

Christian Plagemann,

Wolfram BurgardAuthors Info & Claims

ICML '07: Proceedings of the 24th international conference on Machine learning

Pages 393 - 400

https://doi.org/10.1145/1273496.1273546

Published: 20 June 2007 Publication History

Abstract

This paper presents a novel Gaussian process (GP) approach to regression with input-dependent noise rates. We follow Goldberg et al.'s approach and model the noise variance using a second GP in addition to the GP governing the noise-free output value. In contrast to Goldberg et al., however, we do not use a Markov chain Monte Carlo method to approximate the posterior noise variance but a most likely noise approach. The resulting model is easy to implement and can directly be used in combination with various existing extensions of the standard GPs such as sparse approximations. Extensive experiments on both synthetic and real-world data, including a challenging perception problem in robotics, show the effectiveness of most likely heteroscedastic GP regression.

References

[1]

Bottou, L., & Bengio, Y. (1995). Convergence properties of the kmeans algorithm. Advances in Neural Information Processing Systems. MIT Press.

[2]

Cawley, G., Talbot, N., & Chapelle, O. (2006). Estimating Predictive Variances with Kernel Ridge Regression. Machine Learning Challenges (pp. 56--77).

Digital Library

[3]

Chan, D., Kohn, R., Nott, D., & Kirby, C. (2006). Locally-adaptive semiparametric estimation of the mean and variance functions in regression models. Journal of Computational and Graphical Statistics.

[4]

Ecker, M., & Heltshe, J. (1994). Geostatistical estimates of scallop abundance. In Case studies in biometry, 107--124. John Wiley and Sons.

[5]

Edakunni, N., Schaal, S., & Vijayakumar, S. (2007). Kernel carpentry for online regression using randomly varying coefficient model. Proc. of IJCAI-07.

[6]

Goldberg, P., Williams, C., & Bishop, C. (1998). Regression with input-dependent noise: A gaussian process treatment. NIPS 1998. MIT Press.

Digital Library

[7]

Le, Q., Smola, A., & Canu, S. (2005). Heteroscedastic gp regression. Proc. of ICML-05 (pp. 489--496).

Digital Library

[8]

Nott, D. (1996). Semiparametric estimation of mean and variance functions for non-gaussian data. Computational Statistics, 21, 603--620.

Digital Library

[9]

Opsomer, J., Ruppert, D., Wand, M., Holst, U., & Hossjer, O. (1997). Kriging with nonparametric variance function estimation (Technical Report).

[10]

Rasmussen, C., & Williams, C. (2006). Gaussian processes for machine learning. MIT Press.

Digital Library

[11]

Schölkopf, B., Smola, A. J., Williamson, R. C., & Bartlett, P. L. (2000). New support vector algorithms. Neural Computation, 12, 1207--1245.

Digital Library

[12]

Sigrist, M. (1994). Air Monitoring by Spectroscopic Techniques, vol. 197 of Chemical Analysis Series. John Wiley and Sons.

[13]

Silverman, B. (1985). Some Aspects of the Spline Smoothing Approach to Non-parametric Regression Curve Fitting. Journal of the Royal Statistical Society, 47, 1--52.

[14]

Snelson, E., & Ghahramani, Z. (2006). Variable noise and dimensionality reduction for sparse gaussian processes. Proc. of UAI-06.

[15]

Snelson, E., Rasmussen, C., & Ghahramani, Z. (2003). Warped gaussian processes. NIPS 2003.

Digital Library

[16]

Thrun, S., Burgard, W., & Fox, D. (2005). Probabilistic robotics. MIT Press.

[17]

Williams, C. (1998). Prediction with gaussian processes: From linear regression to linear prediction and beyond. In Learning and inference in graphical models, 599--621. Kluwer Acadamic.

Digital Library

[18]

Williams, P. (1996). Using neural networks to model conditional multivariate densities. Neural Computation, 8, 843--854.

Digital Library

[19]

Yuan, M., & Wahba, G. (2004). Doubly penalized likelihood estimator in heteroscedastic regression. Statistics and Probability Letter, 69, 11--20.

Cited By

Singh JPandey P(2025)Machine learning for materials developments in metals additive manufacturingMachine Learning for Powder-Based Metal Additive Manufacturing10.1016/B978-0-443-22145-3.00003-7(43-75)Online publication date: 2025
https://doi.org/10.1016/B978-0-443-22145-3.00003-7
Lämmle SBogoclu CVoßhall RHaselhoff ARoos DKiyavash NMooij J(2024)Quantifying local model validity using active learningProceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence10.5555/3702676.3702776(2113-2135)Online publication date: 15-Jul-2024
https://dl.acm.org/doi/10.5555/3702676.3702776
Fan MYoon BDougherty EUrban NAlexander FArróyave RQian XKiyavash NMooij J(2024)Multi-fidelity Bayesian optimization with multiple information sources of input-dependent fidelityProceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence10.5555/3702676.3702735(1271-1293)Online publication date: 15-Jul-2024
https://dl.acm.org/doi/10.5555/3702676.3702735
Show More Cited By

Most likely heteroscedastic Gaussian process regression
1. Computing methodologies

Recommendations

Improved Most Likely Heteroscedastic Gaussian Process Regression via Bayesian Residual Moment Estimator
This paper proposes an improved most likely heteroscedastic Gaussian process (MLHGP) algorithm to handle a kind of nonlinear regression problems involving input-dependent noise. The improved MLHGP follows the same learning scheme as the current algorithm ...
Variational Bayesian multinomial probit regression with Gaussian process priors

It is well known in the statistics literature that augmenting binary and polychotomous response models with gaussian latent variables enables exact Bayesian analysis via Gibbs sampling from the parameter posterior. By adopting such a data augmentation ...
Variational Tobit Gaussian Process Regression
Abstract
We propose a variational inference-based framework for training a Gaussian process regression model subject to censored observational data. Data censoring is a typical problem encountered during the data gathering procedure and requires ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '07: Proceedings of the 24th international conference on Machine learning

June 2007

1233 pages

ISBN:9781595937933

DOI:10.1145/1273496

Editor:
Zoubin Ghahramani
University of Cambridge, United Kingdom

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Machine Learning Journal

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ICML '07 & ILP '07

Sponsor:

ICML '07 & ILP '07: The 24th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 20 - 24, 2007

Oregon, Corvalis, USA

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

183
Total Citations
View Citations
1,767
Total Downloads

Downloads (Last 12 months)182
Downloads (Last 6 weeks)18

Reflects downloads up to 15 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Singh JPandey P(2025)Machine learning for materials developments in metals additive manufacturingMachine Learning for Powder-Based Metal Additive Manufacturing10.1016/B978-0-443-22145-3.00003-7(43-75)Online publication date: 2025
https://doi.org/10.1016/B978-0-443-22145-3.00003-7
Lämmle SBogoclu CVoßhall RHaselhoff ARoos DKiyavash NMooij J(2024)Quantifying local model validity using active learningProceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence10.5555/3702676.3702776(2113-2135)Online publication date: 15-Jul-2024
https://dl.acm.org/doi/10.5555/3702676.3702776
Fan MYoon BDougherty EUrban NAlexander FArróyave RQian XKiyavash NMooij J(2024)Multi-fidelity Bayesian optimization with multiple information sources of input-dependent fidelityProceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence10.5555/3702676.3702735(1271-1293)Online publication date: 15-Jul-2024
https://dl.acm.org/doi/10.5555/3702676.3702735
Bravo-Moncayo LPuyana-Romero VArgotti-Gómez MCiaburro G(2024)Enhanced Environmental Sustainability for the Acoustic Absorption Properties of Cabuya Fiber in Building Construction Using Machine Learning Predictive ModelSustainability10.3390/su1614620416:14(6204)Online publication date: 20-Jul-2024
https://doi.org/10.3390/su16146204
Haghi RStagg CCrawford C(2024)Wind Turbine Damage Equivalent Load Assessment Using Gaussian Process Regression Combining Measurement and Synthetic DataEnergies10.3390/en1702034617:2(346)Online publication date: 10-Jan-2024
https://doi.org/10.3390/en17020346
Ungredda JBranke J(2024)Bayesian Optimisation for Constrained ProblemsACM Transactions on Modeling and Computer Simulation10.1145/3641544Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3641544
Bertipaglia AAlirezaei MHappee RShyrokau B(2024)An Unscented Kalman Filter-Informed Neural Network for Vehicle Sideslip Angle EstimationIEEE Transactions on Vehicular Technology10.1109/TVT.2024.338949373:9(12731-12746)Online publication date: Sep-2024
https://doi.org/10.1109/TVT.2024.3389493
Abdulsamad HNickl PKlink PPeters J(2024)Variational Hierarchical Mixtures for Probabilistic Learning of Inverse DynamicsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.331467046:4(1950-1963)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TPAMI.2023.3314670
Tavassoli MKatyara SPozzi MDeshpande NCaldwell DPrattichizzo D(2024)Learning Skills From Demonstrations: A Trend From Motion Primitives to Experience AbstractionIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2023.329616616:1(57-74)Online publication date: Feb-2024
https://doi.org/10.1109/TCDS.2023.3296166
Ding CWang JTu YRen XChen X(2024)Robust parameter design for 3D printing process using stochastic computer modelSimulation Modelling Practice and Theory10.1016/j.simpat.2024.102896132(102896)Online publication date: Apr-2024
https://doi.org/10.1016/j.simpat.2024.102896
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten