Learning Linear Dynamical Systems with Semi-Parametric Least Squares
This provides a robust estimation method for linear systems in fields like control theory and physics, though it appears incremental as it builds on existing semi-parametric frameworks.
The paper tackles the problem of estimating parameters in partially-observed linear dynamical systems with biased, semi-parametric noise, and shows that a prefiltered least squares estimator achieves rates without worst-case dependence on dependency decay and is consistent under marginal stability.
We analyze a simple prefiltered variation of the least squares estimator for the problem of estimation with biased, semi-parametric noise, an error model studied more broadly in causal statistics and active learning. We prove an oracle inequality which demonstrates that this procedure provably mitigates the variance introduced by long-term dependencies. We then demonstrate that prefiltered least squares yields, to our knowledge, the first algorithm that provably estimates the parameters of partially-observed linear systems that attains rates which do not not incur a worst-case dependence on the rate at which these dependencies decay. The algorithm is provably consistent even for systems which satisfy the weaker marginal stability condition obeyed by many classical models based on Newtonian mechanics. In this context, our semi-parametric framework yields guarantees for both stochastic and worst-case noise.