Scalable data-driven modeling of spatio-temporal systems: : Weather forecasting
Abstract
In this paper, a new data-driven method for short-range forecasting of spatio-temporal systems is proposed. It uses NCEP data as raw data to construct forecasting model. The global model consists of several local models. Each local model is constructed in three steps. In the first step, a local dataset is constructed based on NCEP raw data. This dataset is a very high-dimensional data with huge number of redundant and irrelevant features. In the second step, a feature selection method named GRASP is applied on the local dataset and produces a new local dataset whose features are reduced significantly. In the third step, a regression ensemble method called Bagging is used to construct a local model. Both GRASP and Bagging methods are scalable modules with respect to the computational power needed. The proposed method makes it possible to control the trade-off between speed and precision. In addition to the scalability, the proposed method, in some points produces forecasts more precise than the GFS system.
References
[1]
T.J. Anastasio, Data-driven modeling of Alzheimer disease pathogenesis, J Theor Biol 290 (2011), 60–72.
[2]
J. Antoch and D. Hlubinka, Data driven modelling of vertical atmospheric radiation, J Environ Radioact 102 (2011), 1085–1095.
[3]
A.S. Badin, J. Eraifej and S. Greenfield, High-resolution spatio-temporal bioactivity of a novel peptide revealed by optical imaging in rat orbitofrontal cortex in vitro: Possible implications for neurodegenerative diseases, Neuropharmacology 73 (2013), 10–18.
[4]
K. Baker, R.T. Lake, D. Rivet, S. Benston, B. Bommersbach and W. Kirk, Point-trained models in a grid environment: Transforming a potato late blight risk forecast for use with the national digital forecast database, Computers and Electronics in Agriculture 105 (2014), 1–8.
[5]
P. Bermejo, J.A. Gámez and J.M. Puerta, A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets, Pattern Recognition Letters 32 (2011), 701–711.
[6]
L. Breiman, Bagging predictors, Machine Learning 24 (1996), 123–140.
[7]
R. Buizza, M. Leutbecher and L. Isaksen, Potential use of an ensemble of analyses in the ECMWF ensemble prediction system, Quarterly Journal of the Royal Meteorological Society 134 (2008), 2051–2066.
[8]
K. Chakravarty, P. Mukhopadhyay and S. Taraphdar, Cloud microphysical properties as revealed by the CAIPEEX and satellite observations and evaluation of a cloud system resolving model simulation of contrasting large scale environments, Journal of Atmospheric and Solar-Terrestrial Physics 73 (2011), 1790–1797.
[9]
H.K. Chang and W.A. Chien, Neural network with multi-trend simulating transfer function for forecasting typhoon wave, Advances in Engineering Software 37 (2006), 184–194.
[10]
A.L.V. Coelho and D.S.C. Nascimento, On the evolutionary design of heterogeneous Bagging models, Neurocomputing 73 (2010), 3319–3322.
[11]
C. Cuevas and N. García, Improved background modeling for real-time spatio-temporal non-parametric moving object detection strategies, Image and Vision Computing 31 (2013), 616–630.
[12]
P. Dash and A. Ignatov, Validation of clear-sky radiances over oceans simulated with MODTRAN4.2 and global NCEP GDAS fields against nighttime NOAA15–18 and MetOp-A AVHRR data, Remote Sensing of Environment 112 (2008), 3012–3029.
[13]
L. Dong, D. Gong, P.A. Valdes-Sosa, Y. Xia, C. Luo, P. Xu and D. Yao, Simultaneous EEG-fMRI: Trial level spatio-temporal fusion for hierarchically reliable information discovery, Neuro Image 99 (2014), 28–41.
[14]
J. Dubé, M. Thériault and F. Des Rosiers, Commuter rail accessibility and house values: The case of the montreal south shore, Canada, 1992–2009, Transportation Research Part A: Policy and Practice 54 (2013), 49–66.
[15]
G. Everaert, I.S. Pauwels, P. Boets, F. Buysschaert and P.L.M. Goethals, Development and assessment of ecological models in the context of the european water framework directive: Key issues for trainers in data-driven modeling approaches, Ecological Informatics 17 (2013), 111–116.
[16]
F.R. Awal, J. Michaud, P.S. Chu, S. Fares, K. Kodama and M. Rosener, Rainfall-runoff modeling in a flashy tropical watershed using the distributed HL-RDHM model, Journal of Hydrology (2014).
[17]
T.A. Feo and M.G.C. Resende, A probabilistic heuristic for a computationally difficult set covering problem, Operations Research Letters 8 (1989), 67–71.
[18]
T.A. Feo and M.G.C. Resende, Greedy randomized adaptive search procedures, Journal of Global Optimization 6 (1995).
[19]
H.J. Fernando, M.C. Mammarella, G. Grandoni, P. Fedele, R. Di Marco, R. Dimitrova and P. Hyde, Forecasting PM10 in metropolitan areas: Efficacy of neural networks, Environ Pollut 163 (2012), 62–67.
[20]
P. Festa and M.G.C. Resende, Effective application of GRASP, Wiley Encyclopedia of Operations Research and Management Sciences 3 (2011), 1609–1617.
[21]
Filippo, A. Rebelo Torres, B. Kjerfve and A. Monat, Application of artificial neural network (ANN) to improve forecasting of sea level, Ocean & Coastal Management 55 (2012), 101–110.
[22]
M.J. Friedel, Data-driven modeling of surface temperature anomaly and solar activity trends, Environmental Modelling & Software 37 (2012), 217–232.
[23]
Gartner, What is big data, In: University of Villanova, 2012.
[24]
M. Golparvar-Fard, A. Heydarian and J.C. Niebles, Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers, Advanced Engineering Informatics 27 (2013), 652–663.
[25]
Y.C. He, P.W. Chan and Q.S. Li, Standardization of raw wind speed data under complex terrain conditions: A data-driven scheme, Journal of Wind Engineering and Industrial Aerodynamics 131 (2014), 12–30.
[26]
D. Hernández-Lobato, G. Martínez-Muñoz and A. Suárez, Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles, Neurocomputing 74 (2011), 2250–2264.
[27]
D.J. Hill and B.S. Minsker, Anomaly detection in streaming environmental sensor data: A data-driven modeling approach, Environmental Modelling & Software 25 (2010), 1014–1022.
[28]
S. Holly, M.H. Pesaran and T. Yamagata, The spatial and temporal diffusion of house prices in the UK, Journal of Urban Economics 69 (2011), 2–23.
[29]
L.F. Hsiao, M.J. Yang, C.S. Lee, H.C. Kuo, D.S. Shih, C.C. Tsai, C.J. Wang, L.Y. Chang, D.Y.C. Chen, L. Feng, J.S. Hong, C.T. Fong, D.S. Chen, T.C. Yeh, C.Y. Huang, W.D. Guo and G.F. Lin, Ensemble forecasting of typhoon rainfall and floods over a mountainous watershed in Taiwan, Journal of Hydrology 506 (2013), 55–68.
[30]
K. Judd, Forecasting with imperfect models, dynamically constrained inverse problems, and gradient descent algorithms, Physica D: Nonlinear Phenomena 237 (2008), 216–232.
[31]
V. Koren, M. Smith and Z. Cui, Physically-based modifications to the sacramento soil moisture accounting model, Part A: Modeling the effects of frozen ground on the runoff generation process, Journal of Hydrology (2014).
[32]
S.T.K. Lang, M. Leutbecher and S.C. Jones, Impact of perturbation methods in the ECMWF ensemble prediction system on tropical cyclone forecasts, Quarterly Journal of the Royal Meteorological Society 138 (2012), 2030–2046.
[33]
M. Lange, H. Siemen, S. Blome and H.H. Thulke, Analysis of spatio-temporal patterns of African swine fever cases in Russian wild boar does not reveal an endemic situation, Preventive Veterinary Medicine (2014).
[34]
C. Lin, W. Chen, C. Qiu, Y. Wu, S. Krishnan and Q. Zou, LibD3C: Ensemble classifiers with a clustering and dynamic selection strategy, Neurocomputing 123 (2014), 424–435.
[35]
P. Mathiesen, C. Collier and J. Kleissl, A high-resolution, cloud-assimilating numerical weather prediction model for solar irradiance forecasting, Solar Energy 92 (2013), 47–61.
[36]
P. Mathiesen and J. Kleissl, Evaluation of numerical weather prediction for intra-day solar forecasting in the continental United States, Solar Energy 85 (2011), 967–977.
[37]
R. Milne and L. Bennett, Understanding landscape patterns of temporal variability in avian populations to improve environmental impact assessments, Ecological Informatics 14 (2013), 75–78.
[38]
F. Moscone, E. Tosetti and A. Canepa, Real estate market and financial stability in US metropolitan areas: A dynamic model with spatial effects, Regional Science and Urban Economics 49 (2014), 129–146.
[39]
M. Moshki, P. Kabiri and A. Mohebalhojeh, Scalable feature selection in high-dimensional data based on GRASP, Applied Artificial Intelligence 29 (2015), 283–296.
[40]
C.M. Padilla, W. Kihal-Talantikite, V.M. Vieira, P. Rossello, G.L. Nir, D. Zmirou-Navier and S. Deguen, Air quality and social deprivation in four French metropolitan areas – A localized spatio-temporal environmental inequality analysis, Environmental Research 134 (2014), 315–324.
[41]
J.L. Pelly and B.J. Hoskins, How well does the ECMWF ensemble prediction system predict blocking? Quarterly Journal of the Royal Meteorological Society 129 (2003), 1683–1702.
[42]
A. Rasheed, K. Sørli, R. Holdahl and T. Kvamsdal, A multiscale approach to micrositing of wind turbines, Energy Procedia 14 (2012), 1458–1463.
[43]
M.G.C. Resende and C.C. Ribeiro, Greedy randomized adaptive search procedures: Advances, hybridizations, and applications, Handbook of Metaheuristics International Series in Operations Research & Management Science 146 (2010), 283–319.
[44]
L. Rokach, Ensemble-based classifiers, Artificial Intelligence Review 33 (2010), 1–39.
[45]
J.L. Schnase, D.Q. Duffy, G.S. Tamkin, D. Nadeau, J.H. Thompson, C.M. Grieg, M.A. McInerney and W.P. Webster, MERRA analytic services: Meeting the big data challenges of climate science through cloud-enabled climate analytics-as-a-service, Computers, Environment and Urban Systems (2014).
[46]
G. Shaddick and J.V. Zidek, A case study in preferential sampling: Long term monitoring of air pollution in the UK, Spatial Statistic 9 (2014), 51–65.
[47]
D.S. Shih, C.H. Chen and G.T. Yeh, Improving our understanding of flood forecasting using earlier hydro-meteorological intelligence, Journal of Hydrology 512 (2014), 470–481.
[48]
J.E. Stopa and K.F. Cheung, Intercomparison of wind and wave data from the ECMWF reanalysis interim and the NCEP climate forecast system reanalysis, Ocean Modelling 75 (2014), 65–83.
[49]
M.C. Valverde, E. Araujo and H. Campos Velho, Neural network and fuzzy logic statistical downscaling of atmospheric circulation-type specific weather pattern for rainfall forecasting, Applied Soft Computing 22 (2014), 681–694.
[50]
M.C. Valverde Ramírez, H.F. de Campos Velho and N.J. Ferreira, Artificial neural network technique for rainfall forecasting applied to the São Paulo region, Journal of Hydrology 301 (2005), 146–162.
[51]
J.S. Verkade, J.D. Brown, P. Reggiani and A.H. Weerts, Post-processing ECMWF precipitation and temperature ensemble reforecasts for operational hydrologic forecasting at various spatial scales, Journal of Hydrology 501 (2013), 73–91.
[52]
G. Villarini, B.C. Seo, F. Serinaldi and W.F. Krajewski, Spatial and temporal modeling of radar rainfall uncertainties, Atmospheric Research 135–136 (2014), 91–101.
[53]
W. Wei and D.W. Watkins, Data mining methods for hydroclimatic forecasting, Advances in Water Resources 34 (2011), 1390–1400.
[54]
L. Wiegand and P. Knippertz, Equatorward breaking Rossby waves over the north atlantic and mediterranean region in the ECMWF operational ensemble prediction system, Quarterly Journal of the Royal Meteorological Society 140 (2014), 58–71.
[55]
WMO, Definitions of meteorological forecasting ranges, in: WMO (2014).
[56]
Z. Xie, Y. Xu, Q. Hu and P. Zhu, Margin distribution based bagging pruning, Neurocomputing 85 (2012), 11–19.
[57]
Y. Zhang, J. Wang and X. Wang, Review on probabilistic forecasting of wind power generation, Renewable and Sustainable Energy Reviews 32 (2014), 255–270.
Index Terms
- Scalable data-driven modeling of spatio-temporal systems: Weather forecasting
Index terms have been assigned to the content through auto-classification.
Recommendations
Spatio-temporal modeling of climate change impacts on drought forecast using Generative Adversarial Network: A case study in Africa
AbstractDrought is an extreme weather event, affecting the ecological conditions of vegetation and agricultural productivity, poses challenges for millions of people in Africa, and its long-term prediction is definitely important. Accurate drought ...
Highlights- Proposes an advanced GAN forecasting model based on CNN-LSTM for agricultural drought.
- Investigates the impacts of climate change on spatial and temporal drought variation.
- Identifies multivariate remote sensing data over Africa ...
Enhancing short-term forecasting of daily precipitation using numerical weather prediction bias correcting with XGBoost in different regions of China
AbstractAccurate precipitation (P) short-term forecasts are important for engineering studies and water allocation. This study evaluated a method for bias correction of the Numerical Weather Prediction (NWP) of Global Ensemble Forecast System ...
Highlights- The XGBoost is used to downscale and bias-correct the NWP of Global Ensemble Forecast System V2 to forecast short-term P.
Data-Driven Time Series Forecasting for Social Studies Using Spatio-Temporal Graph Neural Networks
GoodIT '21: Proceedings of the Conference on Information Technology for Social GoodTime series forecasting with additional spatial information has attracted a tremendous amount of attention in recent research, due to its importance in various real-world applications on social studies, such as conflict prediction and pandemic ...
Comments
Information & Contributors
Information
Published In
IOS Press and the authors. All rights reserved.
Publisher
IOS Press
Netherlands
Publication History
Published: 01 January 2017
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in