Abstract
To compute the exact solution of Gaussian process regression one needs \(\mathcal{O}(N^3)\) computations for direct and \(\mathcal{O}(N^2)\) for iterative methods since it involves a densely populated kernel matrix of size N ×N, here N denotes the number of data. This makes large scale learning problems intractable by standard techniques.
We propose to use an alternative approach: the kernel matrix is replaced by a data-sparse approximation, called an \({\mathcal H}^2\)-matrix. This matrix can be represented by only \({\cal O}(N m)\) units of storage, where m is a parameter controlling the accuracy of the approximation, while the computation of the \({\mathcal H}^2\)-matrix scales with \({\cal O}(N m \log N)\).
Practical experiments demonstrate that our scheme leads to significant reductions in storage requirements and computing times for large data sets in lower dimensional spaces.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate gaussian process regression. J. of Machine Learning Research 6, 1935–1959 (2005)
Shen, Y., Ng, A., Seeger, M.: Fast gaussian process regression using kd-trees. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) NIPS 18, MIT Press, Cambridge (2006)
Freitas, N.D., Wang, Y., Mahdaviani, M., Lang, D.: Fast krylov methods for n-body learning. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, MIT Press, Cambridge, MA (2006)
Lang, D., Klaas, M., de Freitas, N.: Empirical testing of fast kernel density estimation algorithms. Technical Report TR-2005-03, Department of Computer Science, University of British Columbia (2005)
Börm, S., Grasedyck, L., Hackbusch, W.: Hierarchical Matrices. Lecture Note 21 of the Max Planck Institute for Mathematics in the Sciences (2003)
Grasedyck, L., Hackbusch, W.: Construction and arithmetics of \({\mathcal{H}}\)-matrices. Computing 70(4), 295–334 (2003)
Hackbusch, W., Khoromskij, B., Sauter, S.A.: On \(\mathcal{H}^2\)-matrices. In: Bungartz, H., Hoppe, R., Zenger, C. (eds.) Lect. on Applied Mathematics, pp. 9–29. Springer, Heidelberg (2000)
Börm, S., Hackbusch, W.: Data-sparse approximation by adaptive \({\mathcal{H}}^2\)-matrices. Computing 69, 1–35 (2002)
Rifkin, R., Yeo, G., Poggio, T.: Regularized least-squares classification. In: Suykens, J., Horvath, G., Basu, S., Micchelli, C., Vandewalle, J. (eds.) Advances in Learning Theory: Methods, Models and Applications, pp. 131–153. IOS Press, Amsterdam (2003)
Greenbaum, A.: Iterative methods for solving linear systems, Philadelphia, PA, USA (1997)
Bebendorf, M.: Effiziente numerische Lösung von Randintegralgleichungen unter Verwendung von Niedrigrang-Matrizen. PhD thesis, Uni. Saarbrücken (2000)
Börm, S., Grasedyck, L.: HLib – a library for \({\mathcal H}\)- and \({\mathcal H}^2\)-matrices (1999), Available at http://www.hlib.org/
Börm, S., Grasedyck, L.: Low-rank approximation of integral operators by interpolation. Computing 72, 325–332 (2004)
Grasedyck, L.: Adaptive recompression of \({\mathcal H}\)-matrices for BEM. Computing 74(3), 205–223 (2004)
Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins U. P. (1996)
Buonadonna, P., Hellerstein, J., Hong, W., Gay, D., Madden, S.: Task: Sensor network in a box. In: Proc. of European Workshop on Sensor Networks (2005)
Ng, A.Y., Coates, A., Diel, M., Ganapathi, V., Schulte, J., Tse, B., Berger, E., Liang, E.: Autonomous inverted helicopter flight via reinforcement learning. In: International Symposium on Experimental Robotics (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Börm, S., Garcke, J. (2007). Approximating Gaussian Processes with \({\cal H}^2\)-Matrices. In: Kok, J.N., Koronacki, J., Mantaras, R.L.d., Matwin, S., Mladenič, D., Skowron, A. (eds) Machine Learning: ECML 2007. ECML 2007. Lecture Notes in Computer Science(), vol 4701. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74958-5_8
Download citation
DOI: https://doi.org/10.1007/978-3-540-74958-5_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74957-8
Online ISBN: 978-3-540-74958-5
eBook Packages: Computer ScienceComputer Science (R0)