Bayesian and regularization approaches to multivariable linear system identification: the role of rank penalties
This work addresses the bias/variance trade-off in system identification for control and signal processing applications, offering an incremental improvement over prior regularization strategies.
The paper tackles the problem of linear system identification for MIMO systems by introducing an impulse response estimator that combines ℓ₂ regularization with a rank penalty derived from the log-det heuristic, which accounts for smoothness, stability, and channel coupling. Experiments show it outperforms existing methods based on classic ℓ₂ regularization, atomic norm, and nuclear norm.
Recent developments in linear system identification have proposed the use of non-parameteric methods, relying on regularization strategies, to handle the so-called bias/variance trade-off. This paper introduces an impulse response estimator which relies on an $\ell_2$-type regularization including a rank-penalty derived using the log-det heuristic as a smooth approximation to the rank function. This allows to account for different properties of the estimated impulse response (e.g. smoothness and stability) while also penalizing high-complexity models. This also allows to account and enforce coupling between different input-output channels in MIMO systems. According to the Bayesian paradigm, the parameters defining the relative weight of the two regularization terms as well as the structure of the rank penalty are estimated optimizing the marginal likelihood. Once these hyperameters have been estimated, the impulse response estimate is available in closed form. Experiments show that the proposed method is superior to the estimator relying on the "classic" $\ell_2$-regularization alone as well as those based in atomic and nuclear norm.