Space-Filling Regularization for Robust and Interpretable Nonlinear State Space Models
This work addresses a specific issue in system identification for nonlinear state space models, offering an incremental improvement for researchers and practitioners in control systems and machine learning.
The paper tackles the problem of state trajectory deformation in nonlinear state space models, which leads to poor data coverage and hinders training, interpretability, and robustness. It proposes a space-filling regularization method that improves data distribution in state space, demonstrated on a local model network architecture and a benchmark system, resulting in enhanced interpretability and robustness.
The state space dynamics representation is the most general approach for nonlinear systems and often chosen for system identification. During training, the state trajectory can deform significantly leading to poor data coverage of the state space. This can cause significant issues for space-oriented training algorithms which e.g. rely on grid structures, tree partitioning, or similar. Besides hindering training, significant state trajectory deformations also deteriorate interpretability and robustness properties. This paper proposes a new type of space-filling regularization that ensures a favorable data distribution in state space via introducing a data-distribution-based penalty. This method is demonstrated in local model network architectures where good interpretability is a major concern. The proposed approach integrates ideas from modeling and design of experiments for state space structures. This is why we present two regularization techniques for the data point distributions of the state trajectories for local affine state space models. Beyond that, we demonstrate the results on a widely known system identification benchmark.