Abstract
Multi-task learning models using Gaussian processes (GP) have been recently developed and successfully applied in various applications. The main difficulty with this approach is the computational cost of inference using the union of examples from all tasks. The paper investigates this problem for the grouped mixed-effect GP model where each individual response is given by a fixed-effect, taken from one of a set of unknown groups, plus a random individual effect function that captures variations among individuals. Such models have been widely used in previous work but no sparse solutions have been developed. The paper presents the first sparse solution for such problems, showing how the sparse approximation can be obtained by maximizing a variational lower bound on the marginal likelihood, generalizing ideas from single-task Gaussian processes to handle the mixed-effect model as well as grouping. Experiments using artificial and real data validate the approach showing that it can recover the performance of inference with the full sample, that it outperforms baseline methods, and that it outperforms state of the art sparse solutions for other multi-task GP formulations.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Álvarez, M.A., Lawrence, N.D.: Computationally efficient convolved multiple output Gaussian processes. JMLR 12, 1425–1466 (2011)
Álvarez, M.A., Luengo, D., Titsias, M.K., Lawrence, N.D.: Efficient multioutput Gaussian processes through variational inducing kernels. In: AISTATS (2010)
Álvarez, M., Rosasco, L., Lawrence, N.: Kernels for vector-valued functions: a review. Arxiv preprint arXiv:1106.6251 (2011)
Bonilla, E., Chai, K.M., Williams, C.: Multi-task Gaussian process prediction. In: NIPS, vol. 20, pp. 153–160 (2008)
Lu, Z., Leen, T., Huang, Y., Erdogmus, D.: A reproducing kernel Hilbert space framework for pairwise time series distances. In: ICML, pp. 624–631 (2008)
Pillonetto, G., De Nicolao, G., Chierici, M., Cobelli, C.: Fast algorithms for nonparametric population modeling of large data sets. Automatica 45(1), 173–179 (2009)
Pillonetto, G., Dinuzzo, F., De Nicolao, G.: Bayesian Online Multitask Learning of Gaussian Processes. IEEE T-PAMI 32(2), 193–205 (2010)
Protopapas, P., Giammarco, J.M., Faccioli, L., Struble, M.F., Dave, R., Alcock, C.: Finding outlier light curves in catalogues of periodic variable stars. Monthly Notices of the Royal Astronomical Society 369, 677–696 (2006)
Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate gaussian process regression. The Journal of Machine Learning Research 6, 1939–1959 (2005)
Rasmussen, C.E., Nickisch, H.: Gaussian Processes for Machine Learning (GPML) Toolbox. JMLR 11, 3011–3015 (2010)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press (2005)
Seeger, C., Williams, M., Lawrence, N.: Fast forward selection to speed up sparse gaussian process regression. In: AISTATS 9 (2003)
Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: NIPS, vol. 18, pp. 1257–1264 (2006)
Soszynski, I., Udalski, A., Szymanski, M.: The Optical Gravitational Lensing Experiment. Catalog of RR Lyr Stars in the Large Magellanic Cloud 06. Acta Astronomica 53, 93–116 (2003)
Titsias, M.K.: Variational learning of inducing variables in sparse gaussian processes. In: AISTATS (2009)
Vicini, P., Cobelli, C.: The iterative two-stage population approach to ivgtt minimal modeling: improved precision with reduced sampling. American Journal of Physiology-Endocrinology and Metabolism 280(1), E179 (2001)
Wang, Y., Khardon, R., Protopapas, P.: Shift-invariant grouped multi-task learning for Gaussian processes. In: ECML, pp. 418–434 (2010)
Yu, K., Tresp, V., Schwaighofer, A.: Learning Gaussian processes from multiple tasks. In: ICML, pp. 1012–1019 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, Y., Khardon, R. (2012). Sparse Gaussian Processes for Multi-task Learning. In: Flach, P.A., De Bie, T., Cristianini, N. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2012. Lecture Notes in Computer Science(), vol 7523. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33460-3_51
Download citation
DOI: https://doi.org/10.1007/978-3-642-33460-3_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33459-7
Online ISBN: 978-3-642-33460-3
eBook Packages: Computer ScienceComputer Science (R0)