Regularized parametric system identification: a decision-theoretic formulation
For researchers in system identification, this provides a principled framework that combines the benefits of classical and regularized methods, though it is an incremental theoretical contribution.
This paper develops a decision-theoretic formulation for parametric system identification that bridges classical prediction error methods and regularized approaches, yielding a method robust to small sample sizes and overparameterization.
Parametric prediction error methods constitute a classical approach to the identification of linear dynamic systems with excellent large-sample properties. A more recent regularized approach, inspired by machine learning and Bayesian methods, has also gained attention. Methods based on this approach estimate the system impulse response with excellent small-sample properties. In several applications, however, it is desirable to obtain a compact representation of the system in the form of a parametric model. By viewing the identification of such models as a decision, we develop a decision-theoretic formulation of the parametric system identification problem that bridges the gap between the classical and regularized approaches above. Using the output-error model class as an illustration, we show that this decision-theoretic approach leads to a regularized method that is robust to small sample-sizes as well as overparameterization.