Duality Theory for Non-Markovian Linear Gaussian Models
This provides a more efficient solution for filtering in non-Markovian linear Gaussian models, which is incremental as it builds on existing duality and control theory.
The paper tackles the problem of filtering in partially observed linear Gaussian models with non-Markovian structures by developing a duality theory, resulting in a novel linear predictor called the dual filter that reduces computational complexity from O(T^3) to linear scaling in time horizon T.
This work develops a duality theory for partially observed linear Gaussian models in discrete time. The state process evolves according to a causal but non-Markovian (or higher-order Gauss-Markov) structure, captured by a lower-triangular transition operator, which is related to transformer, with $T$ as the context length. The main contributions are: (i) a dual control system for the linear Gaussian model, formulated as a backward difference equation (B $Î$ E); (ii) a duality principle establishing that a specific linear-quadratic optimal control problem for the B $Î$ E is dual to the filtering problem for the partially observed model; and (iii) an explicit optimal control formula yielding a novel (transformer-like) linear predictor, referred to as the dual filter, whose computational complexity scales linearly in the time horizon $T$, in contrast to the $O(T^3)$ cost of classical smoothing and Wiener-Hopf approaches.