Robust Hierarchical-Optimization RLS Against Sparse Outliers
This addresses robust linear regression for applications where sparse outliers contaminate data, but it appears incremental as it builds on the recently introduced HO-RLS framework.
The paper tackles the problem of outliers in linear-regression models by developing a robust hierarchical-optimization recursive least squares (HO-RLS) method that estimates outliers as nuisance variables using sparsity-inducing regularization. The method shows notable improvements over state-of-the-art techniques in numerical tests on synthetic data, with theoretical convergence guarantees and no need for matrix inversion.
This paper fortifies the recently introduced hierarchical-optimization recursive least squares (HO-RLS) against outliers which contaminate infrequently linear-regression models. Outliers are modeled as nuisance variables and are estimated together with the linear filter/system variables via a sparsity-inducing (non-)convexly regularized least-squares task. The proposed outlier-robust HO-RLS builds on steepest-descent directions with a constant step size (learning rate), needs no matrix inversion (lemma), accommodates colored nominal noise of known correlation matrix, exhibits small computational footprint, and offers theoretical guarantees, in a probabilistic sense, for the convergence of the system estimates to the solutions of a hierarchical-optimization problem: Minimize a convex loss, which models a-priori knowledge about the unknown system, over the minimizers of the classical ensemble LS loss. Extensive numerical tests on synthetically generated data in both stationary and non-stationary scenarios showcase notable improvements of the proposed scheme over state-of-the-art techniques.