Robust High-Dimensional Linear Regression
This work addresses the problem of adversarial data manipulation in supervised learning for high-dimensional regression, offering a more robust solution with practical performance gains, though it is incremental in building on existing robust techniques.
The paper tackles robust linear regression in high-dimensional settings under adversarial data poisoning, relaxing strong prior assumptions to only require that the feature matrix is well-approximated by a low-rank matrix. The proposed method integrates robust low-rank matrix approximation and robust principal component regression, achieving significant improvements in running time and prediction error compared to state-of-the-art methods.
The effectiveness of supervised learning techniques has made them ubiquitous in research and practice. In high-dimensional settings, supervised learning commonly relies on dimensionality reduction to improve performance and identify the most important factors in predicting outcomes. However, the economic importance of learning has made it a natural target for adversarial manipulation of training data, which we term poisoning attacks. Prior approaches to dealing with robust supervised learning rely on strong assumptions about the nature of the feature matrix, such as feature independence and sub-Gaussian noise with low variance. We propose an integrated method for robust regression that relaxes these assumptions, assuming only that the feature matrix can be well approximated by a low-rank matrix. Our techniques integrate improved robust low-rank matrix approximation and robust principle component regression, and yield strong performance guarantees. Moreover, we experimentally show that our methods significantly outperform state of the art both in running time and prediction error.