The goal of the Data Assimilation Office is to produce accurate gridded datasets of atmospheric f... more The goal of the Data Assimilation Office is to produce accurate gridded datasets of atmospheric fields by assimilating a range of observations along with physi-cally consistent model forecasts. The DAO produces datasets that are used by the climate research community. ...
Steven Parker and Rob Armstrong and David Bernholdt and Tamara Dahlgren and Tom Epperly and Josep... more Steven Parker and Rob Armstrong and David Bernholdt and Tamara Dahlgren and Tom Epperly and Joseph Kenny and Manoj Krishnan and Gary Kumfert and Jay Larson and Lois Curfman McInnes and Jarek Nieplocha and Jaideep Ray and Sveta Shasharina. Enabling ...
Symbolic dynamics1 is the study of sequences of symbols belonging to a discrete set of elements, ... more Symbolic dynamics1 is the study of sequences of symbols belonging to a discrete set of elements, the most commmon example being a sequence of ones and zeroes. Often the set of symbols is derived from a timeseries of a continuous variable through the introduction of a partition function--a process called symbolization. Symbolic dynamics has been used widely in the physical sciences; a geophysical example being the application of C1 and C2 complexity2 to hourly precipitation station data3. The C1 and C2 complexities are computed by examining subsequences--or words--of fixed length L in the limit of large values of L. Recent advances in information theory have led to techniques focused on the growth rate of the Shannon entropy and its asymptotic behavior in the limit of long words--levels of entropy convergence4. The result is a set of measures one can use to quantify the amount of memory stored in the sequence, whether or not an observer is able to synchronize to the sequence, and with what confidence it may be predicted. These techniques may also be used to uncover periodic behavior in the sequence. We are currently applying complexity theory and levels of entropy convergence to gridpoint timeseries from the NCAR/NCEP 50-year reanalysis5. Topics to be discussed include: a brief introduction to symbolic dynamics; a description of the partition function/symbolization strategy; a discussion of C1 and C2 complexity and entropy convergence rates and their utility; and example applications of these techniques to NCAR/NCEP 50-reanalyses gridpoint timeseries, resulting in maps of C1 and C2 complexities and entropy convergence rates. Finally, we will discuss how these results may be used to validate climate models. 1{Hao, Bai-Lin, Elementary Symbolic Dynamics and Chaos in Dissipative Systems, Wold Scientific, Singapore (1989)} 2{d'Alessandro, G. and Politi, A., Phys. Rev. Lett., 64, 1609-1612 (1990).} 3{Elsner, J. and Tsonis, A., J. Atmos. Sci., 50, 400-405 (1993).} 4{Crutchfield, J. and Feldman, D., Chaos, {bf 13}, 25-54 (2003).} 5{Kalnay, E.~, Kanamitsu, M.~, Kistler, R.~, Collins, W.~, Deaven, D.~, Gandin, L.~, Iredell, M.~, Saha, S.~, White, G.~, Woolen, J.~, Zhu, Y.~, Chelliah, M.~, Ebisuzaki, W.~, Higgins, W.~, Janowiak, J.~, Mo, K.~C.~, Ropelewski, C.~, Wang, J.~, Leetmaa, A.~, Reynolds, R.~, Jenne, R.~, and Joseph, D.~, Bull. Amer. Met. Soc., 77, 437-471 (1996).}
This paper continues to develop a fault tolerant extension of the sparse grid combination techniq... more This paper continues to develop a fault tolerant extension of the sparse grid combination technique recently proposed in [B. Harding and M. Hegland, ANZIAM J., 54 (CTAC2012), pp. C394-C411]. The approach is novel for two reasons, first it provides several levels in which one can exploit parallelism leading towards massively parallel implementations, and second, it provides algorithm-based fault tolerance so that solutions can still be recovered if failures occur during computation. We present a generalisation of the combination technique from which the fault tolerant algorithm is a consequence. Using a model for the time between faults on each node of a high performance computer we provide bounds on the expected error for interpolation with this algorithm. Numerical experiments on the scalar advection PDE demonstrate that the algorithm is resilient to faults on a real application. It is observed that the trade-off of recovery time to decreased accuracy of the solution is suitably small. A comparison with traditional checkpoint-restart methods applied to the combination technique show that our approach is highly scalable with respect to the number of faults.
2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, 2013
ABSTRACT Computationally-demanding, parallel coupled models are crucial to understanding many imp... more ABSTRACT Computationally-demanding, parallel coupled models are crucial to understanding many important multi-physics/multiscale phenomena. Load-balancing such simulation son large clusters is often done through off-line, static means that often require significant manual input. Dynamic, runtime load-balancing has been shown in our previous work to be effective, but we still used a manually generated performance predictor to guide the load-balancing decisions. In this paper, we show how timing and interaction information obtained by instrumenting the middleware can be used to automatically generate a performance predictor that relates the overall execution time to the execution time of each individual sub model. The performance predictor is evaluated through the new coupled model benchmark employing five constituent sub models that simulates the CCSM coupled climate model.
This article is a theoretical basis for the software implementation of the PhysicalspaceStatistic... more This article is a theoretical basis for the software implementation of the PhysicalspaceStatistical Analysis System (PSAS) that is used for atmospheric data analysis atthe NASA Data Assimilation Office (DAO). The PSAS implements a statistical algorithmthat combines irregularly spaced observations with a gridded forecast to producean optimal estimate of the state of the atmosphere. Starting from models for the forecastand observation
Abstract. Multiphysics and multiscale simulation systems are emerg-ing as a new grand challenge i... more Abstract. Multiphysics and multiscale simulation systems are emerg-ing as a new grand challenge in computational science, largely because of increased computing power provided by the distributed-memory par-allel programming model on commodity clusters. These systems ...
Lecture Notes in Computational Science and Engineering, 2006
Page 1. 10 Parallel PDE-Based Simulations Using the Common Component Architecture Lois Curfman Mc... more Page 1. 10 Parallel PDE-Based Simulations Using the Common Component Architecture Lois Curfman McInnes 1 , Benjamin A. Allan 2 , Robert Armstrong 2 , Steven J. Benson 1 , David E. Bernholdt 3 , Tamara ...
The goal of the Data Assimilation Office is to produce accurate gridded datasets of atmospheric f... more The goal of the Data Assimilation Office is to produce accurate gridded datasets of atmospheric fields by assimilating a range of observations along with physi-cally consistent model forecasts. The DAO produces datasets that are used by the climate research community. ...
Steven Parker and Rob Armstrong and David Bernholdt and Tamara Dahlgren and Tom Epperly and Josep... more Steven Parker and Rob Armstrong and David Bernholdt and Tamara Dahlgren and Tom Epperly and Joseph Kenny and Manoj Krishnan and Gary Kumfert and Jay Larson and Lois Curfman McInnes and Jarek Nieplocha and Jaideep Ray and Sveta Shasharina. Enabling ...
Symbolic dynamics1 is the study of sequences of symbols belonging to a discrete set of elements, ... more Symbolic dynamics1 is the study of sequences of symbols belonging to a discrete set of elements, the most commmon example being a sequence of ones and zeroes. Often the set of symbols is derived from a timeseries of a continuous variable through the introduction of a partition function--a process called symbolization. Symbolic dynamics has been used widely in the physical sciences; a geophysical example being the application of C1 and C2 complexity2 to hourly precipitation station data3. The C1 and C2 complexities are computed by examining subsequences--or words--of fixed length L in the limit of large values of L. Recent advances in information theory have led to techniques focused on the growth rate of the Shannon entropy and its asymptotic behavior in the limit of long words--levels of entropy convergence4. The result is a set of measures one can use to quantify the amount of memory stored in the sequence, whether or not an observer is able to synchronize to the sequence, and with what confidence it may be predicted. These techniques may also be used to uncover periodic behavior in the sequence. We are currently applying complexity theory and levels of entropy convergence to gridpoint timeseries from the NCAR/NCEP 50-year reanalysis5. Topics to be discussed include: a brief introduction to symbolic dynamics; a description of the partition function/symbolization strategy; a discussion of C1 and C2 complexity and entropy convergence rates and their utility; and example applications of these techniques to NCAR/NCEP 50-reanalyses gridpoint timeseries, resulting in maps of C1 and C2 complexities and entropy convergence rates. Finally, we will discuss how these results may be used to validate climate models. 1{Hao, Bai-Lin, Elementary Symbolic Dynamics and Chaos in Dissipative Systems, Wold Scientific, Singapore (1989)} 2{d'Alessandro, G. and Politi, A., Phys. Rev. Lett., 64, 1609-1612 (1990).} 3{Elsner, J. and Tsonis, A., J. Atmos. Sci., 50, 400-405 (1993).} 4{Crutchfield, J. and Feldman, D., Chaos, {bf 13}, 25-54 (2003).} 5{Kalnay, E.~, Kanamitsu, M.~, Kistler, R.~, Collins, W.~, Deaven, D.~, Gandin, L.~, Iredell, M.~, Saha, S.~, White, G.~, Woolen, J.~, Zhu, Y.~, Chelliah, M.~, Ebisuzaki, W.~, Higgins, W.~, Janowiak, J.~, Mo, K.~C.~, Ropelewski, C.~, Wang, J.~, Leetmaa, A.~, Reynolds, R.~, Jenne, R.~, and Joseph, D.~, Bull. Amer. Met. Soc., 77, 437-471 (1996).}
This paper continues to develop a fault tolerant extension of the sparse grid combination techniq... more This paper continues to develop a fault tolerant extension of the sparse grid combination technique recently proposed in [B. Harding and M. Hegland, ANZIAM J., 54 (CTAC2012), pp. C394-C411]. The approach is novel for two reasons, first it provides several levels in which one can exploit parallelism leading towards massively parallel implementations, and second, it provides algorithm-based fault tolerance so that solutions can still be recovered if failures occur during computation. We present a generalisation of the combination technique from which the fault tolerant algorithm is a consequence. Using a model for the time between faults on each node of a high performance computer we provide bounds on the expected error for interpolation with this algorithm. Numerical experiments on the scalar advection PDE demonstrate that the algorithm is resilient to faults on a real application. It is observed that the trade-off of recovery time to decreased accuracy of the solution is suitably small. A comparison with traditional checkpoint-restart methods applied to the combination technique show that our approach is highly scalable with respect to the number of faults.
2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, 2013
ABSTRACT Computationally-demanding, parallel coupled models are crucial to understanding many imp... more ABSTRACT Computationally-demanding, parallel coupled models are crucial to understanding many important multi-physics/multiscale phenomena. Load-balancing such simulation son large clusters is often done through off-line, static means that often require significant manual input. Dynamic, runtime load-balancing has been shown in our previous work to be effective, but we still used a manually generated performance predictor to guide the load-balancing decisions. In this paper, we show how timing and interaction information obtained by instrumenting the middleware can be used to automatically generate a performance predictor that relates the overall execution time to the execution time of each individual sub model. The performance predictor is evaluated through the new coupled model benchmark employing five constituent sub models that simulates the CCSM coupled climate model.
This article is a theoretical basis for the software implementation of the PhysicalspaceStatistic... more This article is a theoretical basis for the software implementation of the PhysicalspaceStatistical Analysis System (PSAS) that is used for atmospheric data analysis atthe NASA Data Assimilation Office (DAO). The PSAS implements a statistical algorithmthat combines irregularly spaced observations with a gridded forecast to producean optimal estimate of the state of the atmosphere. Starting from models for the forecastand observation
Abstract. Multiphysics and multiscale simulation systems are emerg-ing as a new grand challenge i... more Abstract. Multiphysics and multiscale simulation systems are emerg-ing as a new grand challenge in computational science, largely because of increased computing power provided by the distributed-memory par-allel programming model on commodity clusters. These systems ...
Lecture Notes in Computational Science and Engineering, 2006
Page 1. 10 Parallel PDE-Based Simulations Using the Common Component Architecture Lois Curfman Mc... more Page 1. 10 Parallel PDE-Based Simulations Using the Common Component Architecture Lois Curfman McInnes 1 , Benjamin A. Allan 2 , Robert Armstrong 2 , Steven J. Benson 1 , David E. Bernholdt 3 , Tamara ...
Uploads