Learning Kalman Policy for Singular Unknown Covariances via Riemannian Regularization
This addresses a fundamental challenge in estimation theory for applications requiring robust filtering under uncertain conditions, though it is incremental as it builds on existing control--estimation duality and policy optimization frameworks.
The paper tackles the problem of learning optimal Kalman filters under unknown and potentially singular noise covariances by formulating it as a stochastic policy optimization problem. The result is a computationally efficient algorithm with non-asymptotic convergence guarantees, showing robustness in challenging singular estimation regimes.
Kalman filtering is a cornerstone of estimation theory, yet learning the optimal filter under unknown and potentially singular noise covariances remains a fundamental challenge. In this paper, we revisit this problem through the lens of control--estimation duality and data-driven policy optimization, formulating the learning of the steady-state Kalman gain as a stochastic policy optimization problem directly from measurement data. Our key contribution is a Riemannian regularization that reshapes the optimization landscape, restoring structural properties such as coercivity and gradient dominance. This geometric perspective enables the effective use of first-order methods under significantly relaxed conditions, including unknown and rank-deficient noise covariances. Building on this framework, we develop a computationally efficient algorithm with a data-driven gradient oracle, enabling scalable stochastic implementations. We further establish non-asymptotic convergence and error guarantees enabled by the Riemannian regularization, quantifying the impact of bias and variance in gradient estimates and demonstrating favorable scaling with problem dimension. Numerical results corroborate the effectiveness of the proposed approach and robustness to the choice of stepsize in challenging singular estimation regimes.