SciML: Open Source Software for Scientific Machine Learning
Advanced Equation Solvers
The library DifferentialEquations.jl is a library for solving ordinary differential equations (ODEs), stochastic differential equations (SDEs), delay differential equations (DDEs), differential-algebraic equations (DAEs), and hybrid differential equations which include multi-scale models and mixtures with agent-based simulations. Other solvers like NonlinearSolve.jl for f(x)=0 rootfinding problems, Optimization.jl for nonlinear optimization, etc. expand SciML to more equation types, offering high features and performance while integrating with machine learning frameworks through differentiability.
Physics-Informed Model Discovery
SciML contains a litany of modules for automating the process of model discovery and fitting. Tools like DiffEqParamEstim.jl and DiffEqBayes.jl provide classical maximum likelihood and Bayesian estimation for differential equation based models, while SciMLSensitivity.jl enables the deep hand-optimized methods for forward and adjoint senstivity (i.e. derivatives) of equation solvers. This enables the training of embedded neural networks inside of differential equations (neural differential equations or universal differential equations) for discovering unknown dynamical equations.
Polyglot Userbase
While the majority of the tooling for SciML is built using the Julia programming language, SciML is committed to ensure that these methodologies can be used throughout the greater scientific community. Tools like diffeqpy and diffeqr bridge the DifferentialEquations.jl solvers to Python and R respectively, and we hope to see many more developments along these lines in the near future.
Compiler-Assisted Model Analysis and Sparsity Acceleration
Scientific models generally have structures like locality which leads to sparsity in the program structures that can be exploited for major performance acceleration. The SciML builds a set of interconnected tools for generating numerical solver code directly on the models that are being simulated. Symbolics.jl can automatically detect the sparsity patterns of Jacobians and Hessians from arbitrary source code, while ModelingToolkit.jl can rewrite differential equation models to re-arrange equations for better stability and automatically parallelize code.
Large Developer Community and Help Channels
With multiple Slack channels with thousands of users, a highly active Zulip chat server, and hundreds of daily questions and answers on Discourse forums, the SciML community has many channels to get help. The developers also frequent these channels, making it easy to pick up and learn.
ML-Assisted Tooling for Model Acceleration
SciML supports the development of the latest ML-accelerated toolsets for scientific machine learning. Methods like Physics-Informed Neural Networks (PINNs) are productionized in the NeuralPDE.jl library, while the Deep BSDE, the Deep Splitting and the MLP methods for solving 1000 dimensional partial differential equations are available in the HighDimPDE.jl library. Surrogate-based acceleration methods are provided by Surrogates.jl. High level tools like DiffEqFlux.jl and DeepEquilibriumNetworks.jl define pre-made implicit deep learning architectures.
Differentiable Scientific Data Structures and Simulators
The SciML ecosystem contains pre-built scientific simulation tools along with data structures for accelerating the development of models. Tools like LabelledArrays.jl and MultiScaleArrays.jl make it easy to build large-scale scientific models, while other tools like NBodySimulator.jl provide full-scale simulation simulators.
Tools for Accelerated Algorithm Development and Research
SciML is an organization dedicated to helping state-of-the-art research in both numerical simulation methods and methodologies in scientific machine learning. Many tools throughout the organization automate the process of benchmarking and testing new methodologies to ensure they are safe and battle tested, both to accelerate the translation of the methods to publications and to users. We invite the larger research community to make use of our tooling like DiffEqDevTools.jl and our large suite of wrapped algorithms for quickly test and deploying new algorithms.
