Robust Linear Regression: Phase-Transitions and Precise Tradeoffs for General Norms
This work addresses the fundamental tradeoff between adversarial robustness and accuracy in linear regression, applicable to general norms and covariance matrices, extending prior research in the field.
The paper investigates the impact of test-time adversarial attacks on linear regression models, determining the optimal robustness achievable while maintaining standard predictive accuracy, with findings empirically confirmed across various settings.
In this paper, we investigate the impact of test-time adversarial attacks on linear regression models and determine the optimal level of robustness that any model can reach while maintaining a given level of standard predictive performance (accuracy). Through quantitative estimates, we uncover fundamental tradeoffs between adversarial robustness and accuracy in different regimes. We obtain a precise characterization which distinguishes between regimes where robustness is achievable without hurting standard accuracy and regimes where a tradeoff might be unavoidable. Our findings are empirically confirmed with simple experiments that represent a variety of settings. This work applies to feature covariance matrices and attack norms of any nature, and extends beyond previous works in this area.