On a class of optimization-based robust estimators
This work addresses the robust estimation problem for parameter matrices, offering a verifiable and potentially tight bound on outlier tolerance, which is a practical improvement over existing unverifiable restricted isometry-based bounds.
The paper proposes a general optimization-based framework for robust regression where observations are corrupted by both sparse large-magnitude noise and bounded dense noise. It provides a verifiable bound on the number of sparse outliers that can be tolerated while recovering the true parameter, and an error bound for the case with additional dense noise.
We consider in this paper the problem of estimating a parameter matrix from observations which are affected by two types of noise components: (i) a sparse noise sequence which, whenever nonzero can have arbitrarily large amplitude (ii) and a dense and bounded noise sequence of "moderate" amount. This is termed a robust regression problem. To tackle it, a quite general optimization-based framework is proposed and analyzed. When only the sparse noise is present, a sufficient bound is derived on the number of nonzero elements in the sparse noise sequence that can be accommodated by the estimator while still returning the true parameter matrix. While almost all the restricted isometry-based bounds from the literature are not verifiable, our bound can be easily computed through solving a convex optimization problem. Moreover, empirical evidence tends to suggest that it is generally tight. If in addition to the sparse noise sequence, the training data are affected by a bounded dense noise, we derive an upper bound on the estimation error.