Numerical Methods and Software for Scientific AI and Scientific Machine Learning
Modeling practice seems to be partitioned into scientific models defined by mechanistic differential equations and machine learning models defined by parameterizations of neural networks. While the ability for interpretable mechanistic models to extrapolate from little information is seemingly at odds with the big data "model-free" approach of neural networks, the next step in scientific progress is to utilize these methodologies together in order to emphasize their strengths while mitigating weaknesses.
Chris' research is defining scientific AI: the integration of domain models (usually differential equations) with artificial intelligence techniques such as machine learning and probabilistic programming. His foundational scientific machine learning (scientific ML) software, DiffEqFlux.jl, allows researchers to train the vast array of differential equation models mixed with neural networks. This allows for nueral ordinary differential equations (neural ODEs), neural stochastic differential equations (neural SDEs), neural delay differential equations (neural DDEs), neural differential-algebraic equations (neural DAEs), neural partial differential equations (neural PDEs), and hybrid equations like neural jump diffusions (nueral jump SDEs). For most of these equations, it is the first software to support neural network integration, and it does so while providing GPU-acceleration and all of the features of DifferentialEquations.jl, meaning stiff equations can be efficiently integrated with all of the state-of-the-art methods (automated sparsity detection and coloring, Jacobian-Free Newton Krylov, etc.).
Chris' research is showcasing how machine learning integrated into differential equations can be used in many ways, including:
- Accelerating the solution of large sparse ODE and DAE systems and their inverse problems
- Automated data-driven discovery of dynamical equations
- Performing nonlinear optimal control via neural networks
Scientific AI as Next Generation Healthcare
Recent biological techniques, such as next generation sequencing, have enabled the gathering of gigantic amounts of data. For example, microfluidics mixed with RNA-sequencing allows for the generation of entire genomic and transcriptomic profiles from each individual cell in a population, giving a way to quantify not only the chemical concentrations within a cell but also how it differs across cells. Electronic devices such as cellphones and wearables are gathering continuous data on individuals, who become patients. With all of this data, a physician checks a few variables, such as your height, weight, and gender, in order to prescribe a dose.
Chris' research is investigating how we can do better. By mixing nonlinear mixed effects models with machine learning and uncertainty quantification techniques, Chris is developing new methodologies to utilize the big data within the Electronic Health Records (EHR) to enable personalized precision medicine. This is showcased in the culmunating software, Pumas, which integrates an entire pharmacometric modeling stack into Julia with new techniques to be a practical Scientific AI for next generation data-driven healthcare.
High Performance Solving of Differential Equations
Differential equations are the foundation of the model simulators in Scientific AI. Chris is the lead developer of the DifferentialEquations.jl suite of solvers, which includes:
- Discrete equations (function maps, discrete stochastic (Gillespie/Markov) simulations)
- Ordinary differential equations (ODEs)
- Split and Partitioned ODEs (Symplectic integrators, IMEX Methods)
- Stochastic ordinary differential equations (SODEs or SDEs)
- Random differential equations (RODEs or RDEs)
- Differential algebraic equations (DAEs)
- Delay differential equations (DDEs)
- Mixed discrete and continuous equations (Hybrid Equations, Jump Diffusions)
- (Stochastic) partial differential equations ((S)PDEs) (with both finite difference and finite element methods)
The well-optimized DifferentialEquations solvers benchmark as the some of the fastest implementations, using classic algorithms and ones from recent research which routinely outperform the "standard" C/Fortran methods, and include algorithms optimized for high-precision and HPC applications. At the same time, it wraps the classic C/Fortran methods, making it easy to switch over to them whenever necessary. It integrates with the Julia package sphere, for example using Juno's progress meter, automatic plotting, built-in interpolations, and wraps other differential equation solvers so that many different methods for solving the equations can be accessed by simply switching a keyword argument. It utilizes Julia's generality to be able to solve problems specified with arbitrary number types (types with units like Unitful, and arbitrary precision numbers like BigFloats and ArbFloats), arbitrary sized arrays (ODEs on matrices), and more. This gives a powerful mixture of speed and productivity features to help you solve and analyze your differential equations faster.
Chris' research continues to advance DifferentialEquations.jl, including research into multi-GPU parallelization techniques for ODE and DAE solvers, adaptive stochastic differential equation integrators, uncertainty quantification of differential equations, and a lot more.