Auto-differentiable data assimilation: Co-learning of states, dynamics, and filtering algorithms
This work addresses the costly parameter tuning and model accuracy issues in data assimilation for practitioners in fields like aerospace engineering and atmospheric science, offering a versatile and customizable approach.
The paper tackles the problem of data assimilation by introducing a framework for jointly learning states, dynamics, and filtering algorithm parameters through auto-differentiable filtering, enabling gradient-based optimization from partial, noisy observations across diverse scientific domains.
Data assimilation algorithms estimate the state of a dynamical system from partial observations, where the successful performance of these algorithms hinges on costly parameter tuning and on employing an accurate model for the dynamics. This paper introduces a framework for jointly learning the state, dynamics, and parameters of filtering algorithms in data assimilation through a process we refer to as auto-differentiable filtering. The framework leverages a theoretically motivated loss function that enables learning from partial, noisy observations via gradient-based optimization using auto-differentiation. We further demonstrate how several well-known data assimilation methods can be learned or tuned within this framework. To underscore the versatility of auto-differentiable filtering, we perform experiments on dynamical systems spanning multiple scientific domains, such as the Clohessy-Wiltshire equations from aerospace engineering, the Lorenz-96 system from atmospheric science, and the generalized Lotka-Volterra equations from systems biology. Finally, we provide guidelines for practitioners to customize our framework according to their observation model, accuracy requirements, and computational budget.