research-article

Open access

Quantifying Uncertainty in Deep Spatiotemporal Forecasting

Authors:

Matteo Chinazzi,

Alessandro Vespignani,

Rose YuAuthors Info & Claims

KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

Pages 1841 - 1851

https://doi.org/10.1145/3447548.3467325

Published: 14 August 2021 Publication History

Abstract

Deep learning is gaining increasing popularity for spatiotemporal forecasting. However, prior works have mostly focused on point estimates without quantifying the uncertainty of the predictions. In high stakes domains, being able to generate probabilistic forecasts with confidence intervals is critical to risk assessment and decision making. Hence, a systematic study of uncertainty quantification (UQ) methods for spatiotemporal forecasting is missing in the community. In this paper, we describe two types of spatiotemporal forecasting problems: regular grid-based and graph-based. Then we analyze UQ methods from both the Bayesian and the frequentist point of view, casting in a unified framework via statistical decision theory. Through extensive experiments on real-world road network traffic, epidemics, and air quality forecasting tasks, we reveal the statistical and computational trade-offs for different UQ methods: Bayesian methods are typically more robust in mean prediction, while confidence levels obtained from frequentist methods provide more extensive coverage over data variations. Computationally, quantile regression type methods are cheaper for a single confidence interval but require re-training for different intervals. Sampling based methods generate samples that can form multiple confidence intervals, albeit at a higher computational cost.

Supplementary Material

MP4 File (quantifying_uncertainty_in_deep_spatiotemporal-dongxia_wu-liyao_gao-38957899-wPTC.mp4)

Presentation video

Download
141.51 MB

References

[1]

Chris Aicher, Yi-An Ma, Nick Foti, and Emily B. Fox. 2017. Stochastic gradient MCMC for state space models. SIAM Journal on Mathematics of Data Science (SIMODS), Vol. 1 (2017), 555--587.

[2]

AM Alaa and M van der Schaar. 2020. Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functions. In ICML .

[3]

Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565 (2016).

[4]

Ross Askanazi, Francis X Diebold, Frank Schorfheide, and Minchul Shin. 2018. On the comparison of interval forecasts. Journal of Time Series Analysis, Vol. 39, 6 (2018), 953--965.

[5]

Duygu Balcan, Vittoria Colizza, Bruno Goncc alves, Hao Hu, José J Ramasco, and Alessandro Vespignani. 2009. Multiscale mobility networks and the spatial spreading of infectious diseases. Proceedings of the National Academy of Sciences, Vol. 106, 51 (2009), 21484--21489.

[6]

Duygu Balcan, Bruno Goncc alves, Hao Hu, José J Ramasco, Vittoria Colizza, and Alessandro Vespignani. 2010. Modeling the spatial spread of infectious diseases: The GLobal Epidemic and Mobility computational model. Journal of computational science, Vol. 1, 3 (2010), 132--145.

[7]

Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight Uncertainty in Neural Network. In ICML. 1613--1622.

[8]

Glenn W Brier. 1950. Verification of forecasts expressed in terms of probability. Monthly weather review, Vol. 78, 1 (1950), 1--3.

[9]

Matteo Chinazzi, Jessica T. Davis, Marco Ajelli, Corrado Gioannini, Maria Litvinova, Stefano Merler, Ana Pastore y Piontti, Kunpeng Mu, Luca Rossi, Kaiyuan Sun, Cécile Viboud, Xinyue Xiong, Hongjie Yu, M. Elizabeth Halloran, Ira M. Longini, and Alessandro Vespignani. 2020. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science, Vol. 368, 6489 (2020), 395--400.

[10]

Noel Cressie and Christopher K Wikle. 2015. Statistics for spatio-temporal data .John Wiley & Sons.

[11]

Jessica T Davis, Matteo Chinazzi, Nicola Perra, Kunpeng Mu, Ana Pastore y Piontti, Marco Ajelli, Natalie E Dean, Corrado Gioannini, Maria Litvinova, Stefano Merler, Luca Rossi, Kaiyuan Sun, Xinyue Xiong, M. Elizabeth Halloran, Ira M Longini, Cécile Viboud, and Alessandro Vespignani. 2020. Estimating the establishment of local transmission and the cryptic phase of the COVID-19 pandemic in the USA. medRxiv (2020).

[12]

Nan Ding, Youhan Fang, Ryan Babbush, Changyou Chen, Robert D Skeel, and Hartmut Neven. 2014. Bayesian sampling using stochastic gradient thermostats. In NIPS . 3203--3211.

[13]

Ensheng Dong, Hongru Du, and Lauren Gardner. 2020. An interactive web-based dashboard to track COVID-19 in real time. The Lancet infectious diseases, Vol. 20, 5 (2020), 533--534.

[14]

M. Dusenberry, G. Jerfel, Y. Wen, Y.-A. Ma, J. Snoek, K. Heller, B. Lakshminarayanan, and D. Tran. 2020. Efficient and scalable Bayesian neural nets with rank-1 factors. In ICML. 9823--9833.

[15]

Bradley Efron and Trevor Hastie. 2016. Computer Age Statistical Inference . Vol. 5. Cambridge University Press.

Digital Library

[16]

S. Fort, H. Hu, and B. Lakshminarayanan. 2019. Deep ensembles: A loss landscape perspective. (2019). arXiv:1912.02757.

[17]

Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In ICML . 1050--1059.

[18]

Yarin Gal, Jiri Hron, and Alex Kendall. 2017. Concrete dropout. In NIPS. 3581--3590.

[19]

Jan Gasthaus, Konstantinos Benidis, Yuyang Wang, Syama Sundar Rangapuram, David Salinas, Valentin Flunkert, and Tim Januschowski. 2019. Probabilistic forecasting with spline quantile function RNNs. In AISTATS 22 . 1901--1910.

[20]

Xu Geng, Yaguang Li, Leye Wang, Lingyu Zhang, Qiang Yang, Jieping Ye, and Yan Liu. 2019. Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 3656--3663.

Digital Library

[21]

Tilmann Gneiting and Adrian E Raftery. 2007. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, Vol. 102, 477 (2007), 359--378.

[22]

Alex Graves. 2011. Practical variational inference for neural networks. In NIPS. 2348--2356.

[23]

Thomas M Hamill and Daniel S Wilks. 1995. A probabilistic forecast contest and the difficulty in assessing short-range forecast uncertainty. Weather and Forecasting, Vol. 10, 3 (1995), 620--631.

[24]

Jonathan Heek and Nal Kalchbrenner. 2019. Bayesian Inference for Large Scale Image Classification. (2019). arXiv:1908.03491.

[25]

IATA, International Air Transport Association. 2021. https://www.iata.org/ https://www.iata.org/.

[26]

Hosagrahar V Jagadish, Johannes Gehrke, Alexandros Labrinidis, Yannis Papakonstantinou, Jignesh M Patel, Raghu Ramakrishnan, and Cyrus Shahabi. 2014. Big data and its technical challenges. Commun. ACM, Vol. 57, 7 (2014), 86--94.

Digital Library

[27]

Durk P Kingma and Prafulla Dhariwal. 2018. Glow: Generative flow with invertible 1x1 convolutions. In NIPS . 10215--10224.

[28]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013).

[29]

Thomas N Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. (2016).

[30]

Danijel Kivaranovic, Kory D Johnson, and Hannes Leeb. 2020. Adaptive, Distribution-Free Prediction Intervals for Deep Networks. In AISTATS . 4346--4356.

[31]

R. Koenker. 2005. Quantile Regression .Cambridge University Press.

[32]

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In NIPS . 6402--6413.

[33]

Stefan Lee, Senthil Purushwalkam, Michael Cogswell, David Crandall, and Dhruv Batra. 2015. Why M heads are better than one: Training a diverse ensemble of deep networks. arXiv:1511.06314 (2015).

[34]

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In ICLR .

[35]

Haoxing Lin, Rufan Bai, Weijia Jia, Xinyu Yang, and Yongjian You. 2020. Preserving Dynamic Attention for Long-Term Spatial-Temporal Prediction. In ACM SIGKDD . 36--46.

[36]

G Liu and S Shuo. 2018. Air quality forecasting using convolutional LSTM.

[37]

Christos Louizos and Max Welling. 2016. Structured and efficient variational deep learning with matrix gaussian posteriors. In ICML . 1708--1716.

[38]

Yi-An Ma, Tianqi Chen, and Emily Fox. 2015. A complete recipe for stochastic gradient MCMC. In NIPS. 2917--2925.

[39]

Yi-An Ma, Nicholas J. Foti, and Emily B. Fox. 2017. Stochastic gradient MCMC Methods for hidden Markov models. In ICML . 2265--2274.

[40]

Wesley J Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. 2019. A simple baseline for Bayesian uncertainty in deep learning. In NeurIPS . 13153--13164.

[41]

James E Matheson and Robert L Winkler. 1976. Scoring rules for continuous probability distributions. Management science, Vol. 22, 10 (1976), 1087--1096.

[42]

Dina Mistry, Maria Litvinova, Matteo Chinazzi, Laura Fumanelli, Marcelo FC Gomes, Syed A Haque, Quan-Hui Liu, Kunpeng Mu, Xinyue Xiong, M Elizabeth Halloran, et almbox. 2020. Inferring high-resolution human mixing patterns for disease modeling. arXiv preprint arXiv:2003.01214 (2020).

[43]

Radford M Neal. 2012. Bayesian learning for neural networks . Vol. 118. Springer Science & Business Media.

[44]

OAG, Aviation Worlwide Limited. 2021. http://www.oag.com/ http://www.oag.com/.

[45]

Ian Osband, Charles Blundell, Alexander Pritzel, and Benjamin Van Roy. 2016. Deep Exploration via Bootstrapped DQN. In NIPS. 4026--4034.

[46]

Tim Pearce, Alexandra Brintrup, Mohamed Zaki, and Andy Neely. 2018. High-quality prediction intervals for deep learning: A distribution-free, ensembled approach. In ICML. PMLR, 4075--4084.

[47]

Xin Qiu, Elliot Meyerson, and Risto Miikkulainen. 2019. Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel. In ICLR .

[48]

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082 (2014).

[49]

Xiaocheng Shang, Zhanxing Zhu, Benedict Leimkuhler, and Amos J Storkey. 2015. Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian Sampling. In NIPS. 37--45.

[50]

Alexander Shekhovtsov and Boris Flach. 2018. Feed-forward propagation in probabilistic neural networks with categorical and max layers. In ICLR .

[51]

Filippo Simini, Marta C Gonzá lez, Amos Maritan, and Albert-Lá szló Barabá si. 2012. A universal model for mobility and migration patterns . Nature, Vol. 484, 7392 (2012), 96--100.

[52]

Natasa Tagasovska and David Lopez-Paz. 2019. Single-model uncertainties for deep learning. In NeurIPS. 6417--6428.

[53]

Michele Tizzoni, Paolo Bajardi, Chiara Poletto, José J Ramasco, Duygu Balcan, Bruno Goncc alves, Nicola Perra, Vittoria Colizza, and Alessandro Vespignani. 2012. Real-time numerical forecast of global epidemic spreading: case study of 2009 A/H1N1pdm. BMC medicine, Vol. 10, 1 (2012), 165.

[54]

US-EPA. 2012. Revised air quality standards for particle pollution and updates to the Air Quality Index (AQI).

[55]

Thomas Vandal, Evan Kodra, Jennifer Dy, Sangram Ganguly, Ramakrishna Nemani, and Auroop R Ganguly. 2018. Quantifying uncertainty in discrete-continuous and skewed data with Bayesian deep learning. In ACM SIGKDD. 2377--2386.

[56]

Bin Wang, Jie Lu, Zheng Yan, Huaishao Luo, Tianrui Li, Yu Zheng, and Guangquan Zhang. 2019. Deep uncertainty quantification: A machine learning approach for weather forecasting. In ACM SIGKDD . 2087--2095.

[57]

Hao Wang, Xingjian Shi, and Dit-Yan Yeung. 2016. Natural-parameter networks: A class of probabilistic neural networks. NIPS, Vol. 29 (2016), 118--126.

[58]

Yunbo Wang, Mingsheng Long, Jianmin Wang, Zhifeng Gao, and S Yu Philip. 2017. Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms. In NIPS . 879--888.

[59]

Max Welling and Yee W Teh. 2011. Bayesian learning via stochastic gradient Langevin dynamics. In ICML. 681--688.

[60]

Ruofeng Wen, Kari Torkkola, Balakrishnan Narayanaswamy, and Dhruv Madeka. 2017. A multi-horizon quantile recurrent forecaster. arXiv preprint arXiv:1711.11053 (2017).

[61]

Andrew Gordon Wilson and Pavel Izmailov. 2020. Bayesian Deep Learning and a Probabilistic Perspective of Generalization. arXiv:2002.08791 (2020).

[62]

Shi Xingjian, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. 2015. Convolutional LS™ network: A machine learning approach for precipitation nowcasting. In NIPS . 802--810.

[63]

Huaxiu Yao, Xianfeng Tang, Hua Wei, Guanjie Zheng, and Zhenhui Li. 2019. Revisiting spatial-temporal similarity: A deep learning framework for traffic prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 5668--5675.

Digital Library

[64]

Huaxiu Yao, Fei Wu, Jintao Ke, Xianfeng Tang, Yitian Jia, Siyu Lu, Pinghua Gong, Jieping Ye, and Zhenhui Li. 2018. Deep multi-view spatial-temporal network for taxi demand prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.

[65]

Xiuwen Yi, Junbo Zhang, Zhaoyuan Wang, Tianrui Li, and Yu Zheng. 2018. Deep distributed fusion network for air quality prediction. In ACM SIGKDD . 965--973.

[66]

Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 3634--3640.

[67]

Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie. 2019. Why gradient clipping accelerates training: A theoretical justification for adaptivity. arXiv:1905.11881 (2019).

[68]

Qian Zhang, Nicola Perra, Daniela Perrotta, Michele Tizzoni, Daniela Paolotti, and Alessandro Vespignani. 2017a. Forecasting seasonal influenza fusing digital indicators and a mechanistic disease model. In Proceedings of the 26th international conference on world wide web. 311--319.

Digital Library

[69]

Qian Zhang, Kaiyuan Sun, Matteo Chinazzi, Ana Pastore y Piontti, Natalie E Dean, Diana Patricia Rojas, Stefano Merler, Dina Mistry, Piero Poletti, Luca Rossi, et almbox. 2017b. Spread of Zika virus in the Americas. Proceedings of the National Academy of Sciences, Vol. 114, 22 (2017), E4334--E4343.

[70]

Lingxue Zhu and Nikolay Laptev. 2017. Deep and confident prediction for time series at uber. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 103--110.

Cited By

Shiraishi HNakamura TShibuki R(2024)Time Series Quantile Regression Using Random ForestsJournal of Time Series Analysis10.1111/jtsa.1273145:4(639-659)Online publication date: 2-Jan-2024
https://doi.org/10.1111/jtsa.12731
Feng JPiao XLiu HZhang Y(2024)Traffic Anomaly Prediction based on Spatio-Temporal Uncertainty2024 39th Youth Academic Annual Conference of Chinese Association of Automation (YAC)10.1109/YAC63405.2024.10598731(898-903)Online publication date: 7-Jun-2024
https://doi.org/10.1109/YAC63405.2024.10598731
Qian WZhao YZhang DChen BZheng KZhou X(2024)Towards a Unified Understanding of Uncertainty Quantification in Traffic Flow ForecastingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.331226136:5(2239-2256)Online publication date: May-2024
https://doi.org/10.1109/TKDE.2023.3312261
Show More Cited By

Index Terms

Quantifying Uncertainty in Deep Spatiotemporal Forecasting
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by regression
2. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Time series analysis

Recommendations

Quantifying Uncertainty in Discrete-Continuous and Skewed Data with Bayesian Deep Learning
KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Deep Learning (DL) methods have been transforming computer vision with innovative adaptations to other domains including climate change. For DL to pervade Science and Engineering (S&EE) applications where risk management is a core component, well-...
Uncertainty in Bayesian deep label distribution learning
Abstract
Though widely used, most deep convolutional neural networks fail to capture prediction uncertainty, which can be crucial in scenarios such as automotive applications and disease diagnosis. Aleatoric and epistemic uncertainties have ...
Highlights
- We define semantic uncertainty, which complements the aleatoric and model uncertainties, to evaluate the uncertainty related to data incompleteness and ...
Managing uncertainty in spatiotemporal databases

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

August 2021

4259 pages

ISBN:9781450383325

DOI:10.1145/3447548

General Chairs:
Feida Zhu
Singapore Management University
,
Beng Chin Ooi
National University of Singapore
,
Chunyan Miao
Nanyang Technology University
,
Program Chairs:
Haixun Wang,
Iryna Skrypnyk,
Wynne Hsu,
Sanjay Chawla

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2021

Check for updates

Qualifiers

Research-article

Funding Sources

Google Faculty Award
HHS/CDC
AWS Machine Learning Research Award
DARPA
Google Cloud Research Credits program
NSF Grant

Conference

KDD '21

Sponsor:

KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2021

Virtual Event, Singapore

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
2,151
Total Downloads

Downloads (Last 12 months)581
Downloads (Last 6 weeks)77

Reflects downloads up to 02 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shiraishi HNakamura TShibuki R(2024)Time Series Quantile Regression Using Random ForestsJournal of Time Series Analysis10.1111/jtsa.1273145:4(639-659)Online publication date: 2-Jan-2024
https://doi.org/10.1111/jtsa.12731
Feng JPiao XLiu HZhang Y(2024)Traffic Anomaly Prediction based on Spatio-Temporal Uncertainty2024 39th Youth Academic Annual Conference of Chinese Association of Automation (YAC)10.1109/YAC63405.2024.10598731(898-903)Online publication date: 7-Jun-2024
https://doi.org/10.1109/YAC63405.2024.10598731
Qian WZhao YZhang DChen BZheng KZhou X(2024)Towards a Unified Understanding of Uncertainty Quantification in Traffic Flow ForecastingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.331226136:5(2239-2256)Online publication date: May-2024
https://doi.org/10.1109/TKDE.2023.3312261
Mallick TMacfarlane JBalaprakash P(2024)Uncertainty Quantification for Traffic Forecasting Using Deep-Ensemble-Based Spatiotemporal Graph Neural NetworksIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.338109925:8(9141-9152)Online publication date: Aug-2024
https://doi.org/10.1109/TITS.2024.3381099
Laña IOlabarrieta ISer J(2024)Measuring the Confidence of Single-Point Traffic Forecasting Models: Techniques, Experimental Comparison, and Guidelines Toward Their ActionabilityIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.337593625:9(11180-11199)Online publication date: Sep-2024
https://doi.org/10.1109/TITS.2024.3375936
Qian WNielsen TZhao YLarsen KYu J(2024)Uncertainty-Aware Temporal Graph Convolutional Network for Traffic Speed ForecastingIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.336572125:8(8578-8590)Online publication date: Aug-2024
https://doi.org/10.1109/TITS.2024.3365721
Mars Gao LNathan Kutz J(2024)Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constantsProceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences10.1098/rspa.2023.0506480:2286Online publication date: 20-Mar-2024
https://doi.org/10.1098/rspa.2023.0506
Eyring VCollins WGentine PBarnes EBarreiro MBeucler TBocquet MBretherton CChristensen HDagon KGagne DHall DHammerling DHoyer SIglesias-Suarez FLopez-Gomez IMcGraw MMeehl GMolina MMonteleoni CMueller JPritchard MRolnick DRunge JStier PWatt-Meyer OWeigel KYu RZanna L(2024)Pushing the frontiers in climate modelling and analysis with machine learningNature Climate Change10.1038/s41558-024-02095-y14:9(916-928)Online publication date: 23-Aug-2024
https://doi.org/10.1038/s41558-024-02095-y
Sun YPahlavan HChattopadhyay AHassanzadeh PLubis SAlexander MGerber ESheshadri AGuan Y(2024)Data Imbalance, Uncertainty Quantification, and Transfer Learning in Data‐Driven Parameterizations: Lessons From the Emulation of Gravity Wave Momentum Transport in WACCMJournal of Advances in Modeling Earth Systems10.1029/2023MS00414516:7Online publication date: 26-Jul-2024
https://doi.org/10.1029/2023MS004145
Tyralis HPapacharalampous G(2024)A review of predictive uncertainty estimation with machine learningArtificial Intelligence Review10.1007/s10462-023-10698-857:4Online publication date: 18-Mar-2024
https://doi.org/10.1007/s10462-023-10698-8
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents