Abstract
The Integrated Nested Laplace Approximation (INLA) has established itself as a widely used method for approximate inference on Bayesian hierarchical models which can be represented as a latent Gaussian model (LGM). INLA is based on producing an accurate approximation to the posterior marginal distributions of the parameters in the model and some other quantities of interest by using repeated approximations to intermediate distributions and integrals that appear in the computation of the posterior marginals. INLA focuses on models whose latent effects are a Gaussian Markov random field. For this reason, we have explored alternative ways of expanding the number of possible models that can be fitted using the INLA methodology. In this paper, we present a novel approach that combines INLA and Markov chain Monte Carlo (MCMC). The aim is to consider a wider range of models that can be fitted with INLA only when some of the parameters of the model have been fixed. We show how new values of these parameters can be drawn from their posterior by using conditional models fitted with INLA and standard MCMC algorithms, such as Metropolis–Hastings. Hence, this will extend the use of INLA to fit models that can be expressed as a conditional LGM. Also, this new approach can be used to build simpler MCMC samplers for complex models as it allows sampling only on a limited number of parameters in the model. We will demonstrate how our approach can extend the class of models that could benefit from INLA, and how the R-INLA package will ease its implementation. We will go through simple examples of this new approach before we discuss more advanced applications with datasets taken from the relevant literature. In particular, INLA within MCMC will be used to fit models with Laplace priors in a Bayesian Lasso model, imputation of missing covariates in linear models, fitting spatial econometrics models with complex nonlinear terms in the linear predictor and classification of data with mixture models. Furthermore, in some of the examples we could exploit INLA within MCMC to make joint inference on an ensemble of model parameters.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Andrieu, C., Roberts, G.O.: The pseudo-marginal approach to efficient monte carlo computations. Genetics 37(2), 697–725 (2003)
Azzalini, A., Bowman, A.W.: A look at some data on the Old Faithful geyser. Appl. Stat. 39, 357–365 (1990)
Beaumont, M.A.: Estimation of population growth or decline in genetically monitored populations. Genetics 164, 1139–1160 (2003)
Bivand, R.S., Gómez-Rubio, V., Rue, H.: Approximate Bayesian inference for spatial econometrics models. Spat. Stat. 9, 146–165 (2014)
Bivand, R.S., Gómez-Rubio, V., Rue, H.: Spatial data analysis with R-INLA with some extensions. J. Stat. Softw. 63(20), 1–31 (2015)
Chen, M.-H., Shao, Q.-M., Igrahim, J.G.: Monte Carlo Methods in Bayesian Computation. Springer, New York (2000)
Chib, S.: Marginal likelihood from the Gibbs output. J. Am. Stat. Assoc. 90(432), 1313–1321 (1995)
Gilks, W., Gilks, W., Richardson, S., Spiegelhalter, D.: Markov Chain Monte Carlo in Practice. Chapman & Hall, Boca Raton (1996)
Gómez-Rubio, V., Bivand, R.S., Rue, H.: Estimating spatial econometrics models with integrated nested Laplace approximation (2017). arXiv preprint arXiv:1703.01273
Haining, R.: Spatial Data Analysis: Theory and Practice. Cambridge University Press, Cambridge (2003)
Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109 (1970)
Hoeting, J., David Madigan, A.R., Volinsky, C.: Bayesian model averaging: a tutorial. Stat. Sci. 14, 382–401 (1999)
Hubin, A., Storvik, G.: Efficient mode jumping MCMC for Bayesian variable selection in GLMM (2016a). arXiv preprint arXiv:1604.06398
Hubin, A., Storvik, G.: Estimating the marginal likelihood with integrated nested Laplace approximation (INLA) (2016b). arXiv preprint arXiv:1611.01450
James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning with Applications in R. Springer, Berlin (2013)
Joensuu, H., Reichardt, P., Eriksson, M., Hall, K.S., Vehtari, A.: Gastrointestinal stromal tumor: a method for optimizing the timing of CT scans in the follow-up of cancer patients. Radiology 271(1), 96–106 (2014). PMID: 24475826
LeSage, J., Pace, R.K.: Introduction to Spatial Econometrics. Chapman and Hall/CRC, Boca Raton (2009)
Li, Y., Brown, P., Rue, H., Al-Maini, M., Fortin, P.: Spatial modelling of Lupus incidence over 40 years with changes in census areas. J. R. Stat. Soc. Ser. C 61, 99–115 (2012)
Little, R.J.A., Rubin, D.B.: Statistical Analysis with Missing Data. Wiley, Hoboken (2002)
Lykou, A., Ntzoufras, I.: WinBUGS: a tutorial. Wiley Interdiscipl. Rev. Comput. Stat. 3, 385–396 (2011)
Marin, J.-M., Mengersen, K., Robert, C.P.: Bayesian modelling and inference on mixtures of distributions. In: Dey, D.K., Rao, C.R. (eds.) Handbook of Statistics, vol. 25. Elsevier, Amsterdam (2005)
Martins, T.G., Simpson, D., Lindgren, F., Rue, H.: Bayesian computing with INLA: new features. Comput. Stat. Data Anal. 67, 68–83 (2013)
Medina-Aguayo, F.J., Lee, A., Roberts, G.O.: Stability of noisy Metropolis–Hastings. Stat. Comput. 26, 1187–1211 (2016)
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equations of state calculations by fast computing machine. J. Chem. Phys. 21, 1087–1091 (1953)
Pettit, L.I.: The conditional predictive ordinate for the normal distribution. J. R. Stat. Soc. Ser. B (Methodol.) 52(1), 175–184 (1990)
Plummer, M.: rjags: Bayesian Graphical Models using MCMC. R package version 4-6 (2016)
Plummer, M., Best, N., Cowles, K., Vines, K.: CODA: Convergence diagnosis and output analysis for MCMC. R News 6(1), 7–11 (2006)
Rue, H., Martino, S., Chopin, N.: Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion). J. R. Stat. Soc. B 7(2), 319–392 (2009)
Rue, H., Riebler, A., Sørbye, S.H., Illian, J.B., Simpson, D.P., Lindgren, F.K.: Bayesian computing with INLA: a review. Annu. Rev. Stat. Appl. 4, 395–421 (2017)
Schafer, J.L.: Analysis of Incomplete Multivariate Data. Chapman & Hall, London (1997)
Sherlock, C., Thiery, A.H., Roberts, G.O., Rosenthal, J.S.: On the efficiency of pseudo-marginal random walk metropolis algorithms. Ann. Stat. 43(1), 238–275 (2015)
Spiegelhalter, D.J., Best, N.G., Carlin, B.P., Van der Linde, A.: Bayesian measures of model complexity and fit (with discussion). J. R. Stat. Soc. B 64(4), 583–616 (2002)
Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. B 58(1), 267–288 (1996)
Tierney, L., Kadane, J.B.: Accurate approximations for posterior moments and marginal densities. J. Am. Stat. Assoc. 81(393), 82–86 (1986)
van Buuren, S., Groothuis-Oudshoorn, K.: Mice: multivariate imputation by chained equations in R. J. Stat. Softw. 45(1), 1–67 (2011)
Vanhatalo, J., Riihimäki, J., Hartikainen, J., Jylänki, P., Tolvanen, V., Vehtari, A.: GPstuff: Bayesian modeling with Gaussian processes. J. Mach. Learn. Res. 14, 1175–1179 (2013)
Vehtari, A., Mononen, T., Tolvanen, V., Sivula, T., Winther, O.: Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models. J. Mach. Learn. Res. 17(103), 1–38 (2016)
Acknowledgements
Virgilio Gómez-Rubio has been supported by Grant PPIC-2014-001, funded by Consejería de Educación, Cultura y Deportes (JCCM) and FEDER, and Grant MTM2016-77501-P, funded by Ministerio de Economía y Competitividad. We would also like to thank Prof. Aki Vehtari for his comments on a preliminary version of this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gómez-Rubio, V., Rue, H. Markov chain Monte Carlo with the Integrated Nested Laplace Approximation. Stat Comput 28, 1033–1051 (2018). https://doi.org/10.1007/s11222-017-9778-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-017-9778-y