Adaptive Bayesian Optimization for Robust Identification of Stochastic Dynamical Systems
This work addresses the problem of robust system identification for researchers and practitioners in control theory and signal processing, offering an incremental improvement over existing Bayesian optimization methods.
The paper tackles the identification of linear stochastic dynamical systems by proposing an ensemble Gaussian process-based Bayesian optimization method, which outperforms maximum likelihood estimation in terms of RMSE and statistical consistency, achieving RMSE below the classical Cramer-Rao bound for difficult parameters like the inverse time constant.
This paper deals with the identification of linear stochastic dynamical systems, where the unknowns include system coefficients and noise variances. Conventional approaches that rely on the maximum likelihood estimation (MLE) require nontrivial gradient computations and are prone to local optima. To overcome these limitations, a sample-efficient global optimization method based on Bayesian optimization (BO) is proposed, using an ensemble Gaussian process (EGP) surrogate with weighted kernels from a predefined dictionary. This ensemble enables a richer function space and improves robustness over single-kernel BO. Each objective evaluation is efficiently performed via Kalman filter recursion. Extensive experiments across parameter settings and sampling intervals show that the EGP-based BO consistently outperforms MLE via steady-state filtering and expectation-maximization (whose derivation is a side contribution) in terms of RMSE and statistical consistency. Unlike the ensemble variant, single-kernel BO does not always yield such gains, underscoring the benefits of model averaging. Notably, the BO-based estimator achieves RMSE below the classical Cramer-Rao bound, particularly for the inverse time constant, long considered difficult to estimate. This counterintuitive outcome is attributed to a data-driven prior implicitly induced by the GP surrogate in BO.