article

Parallelizing MCMC for Bayesian spatiotemporal geostatistical models

Authors:

Mary Kathryn Cowles,

Marc P. ArmstrongAuthors Info & Claims

Statistics and Computing, Volume 17, Issue 4

Pages 323 - 335

https://doi.org/10.1007/s11222-007-9022-2

Published: 01 December 2007 Publication History

Abstract

When MCMC methods for Bayesian spatiotemporal modeling are applied to large geostatistical problems, challenges arise as a consequence of memory requirements, computing costs, and convergence monitoring. This article describes the parallelization of a reparametrized and marginalized posterior sampling (RAMPS) algorithm, which is carefully designed to generate posterior samples efficiently. The algorithm is implemented using the Parallel Linear Algebra Package (PLAPACK). The scalability of the algorithm is investigated via simulation experiments that are implemented using a cluster with 25 processors. The usefulness of the method is illustrated with an application to sulfur dioxide concentration data from the Air Quality System database of the U.S. Environmental Protection Agency.

References

[1]

Adams, N.M., Kirby, S.P.J., Harris, P., Clegg, D.B.: A review of parallel processing for statistical computation. Stat. Comput. 6, 37-49 (1996).

[2]

Agarwal, D.K., Gelfand, A.E.: Slice sampling for simulation based fitting of spatial data models. Stat. Comput. 15, 61-69 (2002).

Digital Library

[3]

Banerjee, S., Carlin, B.P., Gelfand, A.E.: Hierarchical Modeling and Analysis for Spatial Data. Chapman & Hall/CRC, London (2004).

[4]

Blackford, L.S., Choi, J., Cleary, A., D'Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK Users' Guide. Society for Industrial and Applied Mathematics, Philadelphia (1997).

[5]

Chib, S.: Bayes regression with autoregressive errors. A Gibbs sampling approach. J. Econom. 58, 275-294 (1993).

[6]

Christiansen, C.L., Morris, C.N.: Hierarchical Poisson regression modeling. J. Am. Stat. Assoc. 92, 618-632 (1997).

[7]

Cowles, M.K., Zimmerman, D.L., Christ, A., McGinnis, D.L.: Combining snow water equivalent data from multiple sources to estimate spatio-temporal trends and compare measurement systems. J. Agric. Biol. Environ. Stat. 7, 536-557 (2002).

[8]

Daniels, M.J.: A prior for the variance in hierarchical models. Can. J. Stat. 27, 567-578 (1999).

[9]

Diggle, P., Ribeiro, P.J.: Bayesian inference in Gaussian model-based geostatistics. Geogr. Environ. Model. 6, 129-146 (2002).

[10]

Duff, I.S., van der Vorst, H.A.: Developments and trends in the parallel solution of linear systems. Parallel Comput. 25, 1931-1970 (1999).

Digital Library

[11]

Finley, A.O., Banerjee, S., Carlin, B.P.: spBayes: spBayes Fits Gaussian Models with Potentially Complex Hierarchical Error Structures. R package version 0.0-1 (2006). http://blue.fr.umn. edu/spatialBayes.

[12]

Gilks, W.R., Best, N.G., Tan, K.K.C.: Adaptive rejection Metropolis sampling within Gibbs sampling (Corr: 97V46 p. 541-542 with R.M. Neal). Appl. Stat. 44, 455-472 (1995).

[13]

Kass, R.E., Carlin, B.P., Gelman, A., Neal, R.M.: Markov chain Monte Carlo in practice: a roundtable discussion. Am. Stat. 52, 93-100 (1998).

[14]

Neal, R.M.: Slice sampling. Ann. Stat. 31, 705-767 (2003).

[15]

Nychka, D.: Fields: tools for spatial data. R package version 3.04 (2005). http://www.image.ucar.edu/GSP/Software/Fields.

[16]

Reed, D.A.: Grids, the teragrid, and beyond. Computer 36, 62-68 (2003).

Digital Library

[17]

Rosenthal, J.S.: Parallel computing and Monte Carlo algorithms. Far East J. Theor. Stat. 4, 207-236 (2000).

[18]

Rossini, A., Tierney, L., Li, N.: Simple parallel statistical computing in R. Working paper, UW Biostatistics (2003).

[19]

Rue, H.: Fast sampling of Gaussian Markov random fields. J. Roy. Stat. Soc. Ser. B: Stat. Methodol. 63, 325-338 (2001).

[20]

Rue, H., Martino, S.: Approximate inference for hierarchical Gaussian Markov random field models, Technical report, Department of Mathematical Sciences, NTNU, Norway (2005).

[21]

Rue, H., Tjelmeland, H.: Fitting Gaussian Markov random fields to Gaussian fields. Scand. J. Stat. 29, 31-9 (2002).

[22]

Sargent, D.J., Hodges, J.S., Carlin, B.P.: Structured Markov chain Monte Carlo. J. Comput. Graph. Stat. 9, 217-234 (2000).

[23]

Schervish, M.J.: Applications of parallel computation to statistical inference. J. Am. Stat. Assoc. 83, 976-983 (1988).

[24]

Sylwestrowicz, J.D.: Parallel processing in statistics. In: H. Caussinus, P. Ettinger, R. Tomassone (eds.) COMPSTAT 1982, Proceedings in Computational Statistics, pp. 131-136. Physica-Verlag Ges.m.b.H. (1982).

[25]

van de Geijn, R.A.: Using PLAPACK. MIT Press (1997).

[26]

Whiley, M., Wilson, S.P.: Parallel algorithms for Markov chain Monte Carlo methods in latent spatial Gaussian models. Stat. Comput. 14, 171-179 (2004).

[27]

Wilkinson, D.J.: Parallel Bayesian computation. In: Kontoghiorghes, E.J. (ed.) Handbook of Parallel Computing and Statistics, pp. 481-512. Dekker/CRC Press, New York (2005).

Cited By

Mingas GBouganis C(2016)Population-Based MCMC on Multi-Core CPUs, GPUs and FPGAsIEEE Transactions on Computers10.1109/TC.2015.243925665:4(1283-1296)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1109/TC.2015.2439256
Schliep EHoeting J(2015)Data augmentation and parameter expansion for independent or spatially correlated ordinal dataComputational Statistics & Data Analysis10.1016/j.csda.2015.03.02090:C(1-14)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1016/j.csda.2015.03.020
Padmanabhan AWang SNavarro JTowns JBrown SKatz D(2011)A CyberGIS gateway approach to interoperable access to the National Science Foundation TeraGrid and the Open Science GridProceedings of the 2011 TeraGrid Conference: Extreme Digital Discovery10.1145/2016741.2016786(1-8)Online publication date: 18-Jul-2011
https://dl.acm.org/doi/10.1145/2016741.2016786
Show More Cited By

Index Terms

Parallelizing MCMC for Bayesian spatiotemporal geostatistical models

Recommendations

Simulation-based Bayesian inference for epidemic models

A powerful and flexible method for fitting dynamic models to missing and censored data is to use the Bayesian paradigm via data-augmented Markov chain Monte Carlo (DA-MCMC). This samples from the joint posterior for the parameters and missing data, but ...
Sample selection for MCMC-based recommender systems
RecSys '13: Proceedings of the 7th ACM conference on Recommender systems

Bayesian Inference with Markov Chain Monte Carlo (MCMC) has been shown to provide high prediction quality in recommender systems. The advantage over learning methods such as coordinate descent/alternating least-squares (ALS) or (stochastic) gradient ...
A review of multiple try MCMC algorithms for signal processing
Abstract
Many applications in signal processing require the estimation of some parameters of interest given a set of observed data. More specifically, Bayesian inference needs the computation of a-posteriori estimators which are often expressed ...

Comments

Information & Contributors

Information

Published In

cover image Statistics and Computing

Statistics and Computing Volume 17, Issue 4

December 2007

117 pages

ISSN:0960-3174

Issue’s Table of Contents

Copyright © Copyright © 2007 Springer Science+Business Media, LLC.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 December 2007

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mingas GBouganis C(2016)Population-Based MCMC on Multi-Core CPUs, GPUs and FPGAsIEEE Transactions on Computers10.1109/TC.2015.243925665:4(1283-1296)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1109/TC.2015.2439256
Schliep EHoeting J(2015)Data augmentation and parameter expansion for independent or spatially correlated ordinal dataComputational Statistics & Data Analysis10.1016/j.csda.2015.03.02090:C(1-14)Online publication date: 1-Oct-2015
https://dl.acm.org/doi/10.1016/j.csda.2015.03.020
Padmanabhan AWang SNavarro JTowns JBrown SKatz D(2011)A CyberGIS gateway approach to interoperable access to the National Science Foundation TeraGrid and the Open Science GridProceedings of the 2011 TeraGrid Conference: Extreme Digital Discovery10.1145/2016741.2016786(1-8)Online publication date: 18-Jul-2011
https://dl.acm.org/doi/10.1145/2016741.2016786
Tibbits MHaran MLiechty J(2011)Parallel multivariate slice samplingStatistics and Computing10.1007/s11222-010-9178-z21:3(415-430)Online publication date: 1-Jul-2011
https://dl.acm.org/doi/10.1007/s11222-010-9178-z
Wang SLiu Y(2009)TeraGrid GIScience GatewayInternational Journal of Geographical Information Science10.1080/1365881090275497723:5(631-656)Online publication date: 1-May-2009
https://dl.acm.org/doi/10.1080/13658810902754977
Wang SArmstrong M(2009)A theoretical approach to the use of cyberinfrastructure in geographical analysisInternational Journal of Geographical Information Science10.1080/1365881080191850923:2(169-193)Online publication date: 1-Feb-2009
https://dl.acm.org/doi/10.1080/13658810801918509
Everitt RGlendinning R(2009)A statistical approach to the problem of restoring damaged and contaminated imagesPattern Recognition10.1016/j.patcog.2008.06.00942:1(115-125)Online publication date: 1-Jan-2009
https://dl.acm.org/doi/10.1016/j.patcog.2008.06.009

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents