DiffFind: Discovering Differential Equations from Time Series
Authors: 
Lalithsai Posam, Shubhranshu Shekhar, Meng-Chieh Lee, 
Christos FaloutsosAuthors Info & Claims
Lalithsai Posam, Shubhranshu Shekhar, Meng-Chieh Lee, Christos FaloutsosAuthors Info & ClaimsPages 175 - 187
Abstract
Given one or more time sequences, how can we extract their governing equations? Single and co-evolving time sequences appear in numerous settings, including medicine (neuroscience - EEG signals, cardiology - EKG), epidemiology (covid/flu spreading over time), physics (astrophysics, material science), marketing (sales and competition modeling; market penetration), and numerous more. Linear differential equations will fail, since the underlying equations are often non-linear (SIR model for virus/product spread; Lotka-Volterra for product/species competition, Van der Pol for heartbeat modeling).
We propose DiffFind and we use genetic algorithms to find suitable, parsimonious, differential equations. Thanks to our careful design decisions, DiffFind has the following properties - it is: (a) Effective, discovering the correct model when applied on real and synthetic nonlinear dynamical systems, (b) Explainable, gives succinct differential equations, and (c) Hands-off, requiring no manual hyperparameter specification.
DiffFind outperforms traditional methods (like auto-regression), includes as special case and thus outperforms a recent baseline (‘SINDy’), and wins first or second place for all 5 real and synthetic datasets we tried, often achieving excellent, zero or near-zero RMSE of 0.005.
References
[1]
Anisiu MC Lotka, Volterra and their model Didáctica Mathematica 2014 32 9-17
[2]
Bongard J and Lipson H Automated reverse engineering of nonlinear dynamical systems PNAS 2007 104 24 9943-9948
[3]
Box GE, Jenkins GM, and MacGregor JF Some recent advances in forecasting and control J. Roy. Stat. Soc.: Ser. C (Appl. Stat.) 1974 23 2 158-179
[4]
Brauer, F., Castillo-Chavez, C., Castillo-Chavez, C.: Mathematical Models in Population Biology and Epidemiology, vol. 2. Springer, Cham (2012).
[5]
Brown, R.G.: Statistical forecasting for inventory control (1959)
[6]
Brunton SL, Proctor JL, and Kutz JN Discovering governing equations from data by sparse identification of nonlinear dynamical systems PNAS 2016 113 15 3932-3937
[7]
Chai T and Draxler RR Root Mean Square Error (RMSE) or Mean Absolute Error (MAE)?-arguments against avoiding RMSE in the literature Geosci. Model Dev. 2014 7 3 1247-1250
[8]
Chatfield, C.: Time-Series Forecasting. Chapman and Hall/CRC, Boca Raton (2000)
[9]
Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. In: NeurIPS, vol. 31 (2018)
[10]
Cranmer, M., et al.: Discovering symbolic models from deep learning with inductive biases. In: NeurIPS, vol. 33, pp. 17429–17442 (2020)
[11]
Ding S, Li H, Su C, Yu J, and Jin F Evolutionary artificial neural networks: a review Artif. Intell. Rev. 2013 39 3 251-260
[12]
Elsken T, Metzen JH, and Hutter F Neural architecture search: a survey JMLR 2019 20 1 1997-2017
[13]
FitzHugh R Impulses and physiological states in theoretical models of nerve membrane Biophys. J . 1961 1 6 445-466
[14]
Kramer, O., Kramer, O.: Genetic Algorithms. Springer, Cham (2017)
[15]
Lim B and Zohren S Time-series forecasting with deep learning: a survey Phil. Trans. R. Soc. A 2021 379 2194 20200209
[16]
Lorenz EN Deterministic nonperiodic flow J. Atmos. Sci. 1963 20 2 130-141
[17]
Luo, Y., Xu, C., Liu, Y., Liu, W., Zheng, S., Bian, J.: Learning differential operators for interpretable time series modeling. In: ACM SIGKDD, pp. 1192–1201 (2022)
[18]
Makridakis S and Hibon M ARMA models and the Box-Jenkins methodology J. Forecast. 1997 16 3 147-163
[19]
Park, N., Kim, M., Hoai, N.X., McKay, R.B., Kim, D.K.: Knowledge-based dynamic systems modeling: a case study on modeling river water quality. In: ICDE, pp. 2231–2236. IEEE (2021)
[20]
Van der Pol, B.: LXXXVIII. On “relaxation-oscillations”. The London, Edinburgh, and Dublin Philos. Mag. J. Sci. 2(11), 978–992 (1926)
[21]
Raissi M, Perdikaris P, and Karniadakis GE Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations J. Comput. Phys. 2019 378 686-707
[22]
Sahoo, S., Lampert, C., Martius, G.: Learning equations for extrapolation and control. In: ICML, pp. 4442–4450. PMLR (2018)
[23]
Salinas D, Flunkert V, Gasthaus J, and Januschowski T DeepAR: probabilistic forecasting with autoregressive recurrent networks Int. J. Forecast. 2020 36 3 1181-1191
[24]
Schmidt, M., Lipson, H.: Distilling free-form natural laws from experimental data. Science 324(5923), 81–85 (2009)
[25]
Stanley KO and Miikkulainen R Evolving neural networks through augmenting topologies Evol. Comput. 2002 10 2 99-127
[26]
Tealab A Time series forecasting using artificial neural networks methodologies: a systematic review Future Comput. Inform. J. 2018 3 2 334-340
[27]
Torres JF, Hadjout D, Sebaa A, Martínez-Álvarez F, and Troncoso A Deep learning for time series forecasting: a survey Big Data 2021 9 1 3-21
[28]
Wang, R., Maddix, D., Faloutsos, C., Wang, Y., Yu, R.: Bridging physics-based and data-driven modeling for learning dynamical systems. In: Learning for Dynamics and Control, pp. 385–398. PMLR (2021)
[29]
Wang, R., Robinson, D., Faloutsos, C., Wang, Y.B., Yu, R.: AutoODE: bridging physics-based and data-driven modeling for COVID-19 forecasting. In: NeurIPS 2020 Workshop on Machine Learning in Public Health (2020)
[30]
Zhou X, Qin AK, Gong M, and Tan KC A survey on evolutionary construction of deep neural networks IEEE Trans. Evol. Comput. 2021 25 5 894-912
Recommendations
Numerical simulations of time-dependent partial differential equations
When a time-dependent partial differential equation (PDE) is discretized in space with a spectral approximation, the result is a coupled system of ordinary differential equations (ODEs) in time. This is the notion of the method of lines (MOL), and the ...
Two-dimensional differential transform for partial differential equations
The differential transform is a numerical method for solving differential equations. In this paper, we present the definition and operation of the two-dimensional differential transform. A distinctive feature of the differential transform is its ability ...
Partial differential equations on time scales
Dynamic equations on time scalesDiscrete and continuous formulations of partial differential operators are unified by a time scale formulation of partial differential operators. Results include an Euler-Lagrange equation for double integral variational problems on time scales and a ...
Comments
Information & Contributors
Information
Published In
May 2024
328 pages
ISBN:978-981-97-2265-5
DOI:10.1007/978-981-97-2266-2
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 07 May 2024
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Sep 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in