Kernel Operator-Theoretic Bayesian Filter for Nonlinear Dynamical Systems
This work provides a novel method for real-time machine learning applications, such as streaming data analysis, by offering an adaptive, linear minimum-variance algorithm for tracking nonlinear systems, though it is incremental as it builds on existing Koopman operator theory.
The authors tackled the problem of modeling unknown, nonlinear dynamical systems by proposing a kernel operator-theoretic Bayesian filter that lifts nonlinear dynamics to an infinite-dimensional Hilbert space using RKHS, enabling the application of linear Bayesian methods like the Kalman filter. They demonstrated that this approach achieves accurate results and outperforms finite-dimensional Koopman decomposition, with excellent approximation using a small dimension due to the rapid decay of the Gaussian kernel.
Motivated by the surge of interest in Koopman operator theory, we propose a machine-learning alternative based on a functional Bayesian perspective for operator-theoretic modeling of unknown, data-driven, nonlinear dynamical systems. This formulation is directly done in an infinite-dimensional space of linear operators or Hilbert space with universal approximation property. The theory of reproducing kernel Hilbert space (RKHS) allows the lifting of nonlinear dynamics to a potentially infinite-dimensional space via linear embeddings, where a general nonlinear function is represented as a set of linear functions or operators in the functional space. This allows us to apply classical linear Bayesian methods such as the Kalman filter directly in the Hilbert space, yielding nonlinear solutions in the original input space. This kernel perspective on the Koopman operator offers two compelling advantages. First, the Hilbert space can be constructed deterministically, agnostic to the nonlinear dynamics. The Gaussian kernel is universal, approximating uniformly an arbitrary continuous target function over any compact domain. Second, Bayesian filter is an adaptive, linear minimum-variance algorithm, allowing the system to update the Koopman operator and continuously track the changes across an extended period of time, ideally suited for modern data-driven applications such as real-time machine learning using streaming data. In this paper, we present several practical implementations to obtain a finite-dimensional approximation of the functional Bayesian filter (FBF). Due to the rapid decay of the Gaussian kernel, excellent approximation is obtained with a small dimension. We demonstrate that this practical approach can obtain accurate results and outperform finite-dimensional Koopman decomposition.