Robustly Learning Monotone Single-Index Models
This addresses robust learning of a fundamental model class in machine learning with theoretical guarantees against adversarial noise.
The paper tackles learning monotone single-index models with adversarial label noise under Gaussian distribution, presenting the first computationally efficient algorithm that achieves constant factor approximation for all monotone activations with bounded moment of order 2+ζ, including discontinuous functions like halfspaces.
We consider the basic problem of learning Single-Index Models with respect to the square loss under the Gaussian distribution in the presence of adversarial label noise. Our main contribution is the first computationally efficient algorithm for this learning task, achieving a constant factor approximation, that succeeds for the class of {\em all} monotone activations with bounded moment of order $2 + ζ,$ for $ζ> 0.$ This class in particular includes all monotone Lipschitz functions and even discontinuous functions like (possibly biased) halfspaces. Prior work for the case of unknown activation either does not attain constant factor approximation or succeeds for a substantially smaller family of activations. The main conceptual novelty of our approach lies in developing an optimization framework that steps outside the boundaries of usual gradient methods and instead identifies a useful vector field to guide the algorithm updates by directly leveraging the problem structure, properties of Gaussian spaces, and regularity of monotone functions.