IEnSF: Iterative Ensemble Score Filter for Reducing Error in Posterior Score Estimation in Nonlinear Data Assimilation
For researchers in data assimilation, this work improves the accuracy of score-based diffusion models for nonlinear systems, addressing a known bottleneck.
The paper addresses the structural error in posterior score estimation in the Ensemble Score Filter (EnSF) for nonlinear data assimilation. The proposed iterative ensemble score filter (IEnSF) reduces this error, improving tracking accuracy in high-dimensional dynamical systems.
The Ensemble Score Filter (EnSF) is a score-based diffusion model approach for solving high-dimensional and nonlinear data assimilation problems. While initial applications of EnSF to the Lorenz-96 model and the quasi-geostrophic system showed potential, the current method employs a heuristic weighted sum to combine the prior and the likelihood score functions. This introduces a structural error into the estimation of the posterior score function in the nonlinear setting. This work addresses this challenge by developing an iterative ensemble score filter (IEnSF) that applies an iterative algorithm as an outer loop around the reverse-time stochastic differential equation solver. When the state dynamics or the observation operator is nonlinear, the iterative algorithm can gradually reduce the posterior score estimation error by improving the accuracy of approximating the conditional expectation of the likelihood score function. The number of iterations required depends on the distance between the prior and posterior distributions. Numerical experiments demonstrate that the IEnSF algorithm substantially reduces the error in posterior score estimation in the nonlinear setting and thus improves the accuracy of tracking high-dimensional dynamical systems.