A Fully Polynomial-Time Algorithm for Robustly Learning Halfspaces over the Hypercube
This solves a fundamental problem in robust machine learning for researchers and practitioners, providing efficient learning with discrete distributions where prior methods failed.
The paper presents the first fully polynomial-time algorithm for learning halfspaces on the hypercube under adversarial contamination, achieving an error guarantee of η^O(1) + ε, where η is the noise rate, addressing a long-standing open problem in agnostic learning.
We give the first fully polynomial-time algorithm for learning halfspaces with respect to the uniform distribution on the hypercube in the presence of contamination, where an adversary may corrupt some fraction of examples and labels arbitrarily. We achieve an error guarantee of $η^{O(1)}+ε$ where $η$ is the noise rate. Such a result was not known even in the agnostic setting, where only labels can be adversarially corrupted. All prior work over the last two decades has a superpolynomial dependence in $1/ε$ or succeeds only with respect to continuous marginals (such as log-concave densities). Previous analyses rely heavily on various structural properties of continuous distributions such as anti-concentration. Our approach avoids these requirements and makes use of a new algorithm for learning Generalized Linear Models (GLMs) with only a polylogarithmic dependence on the activation function's Lipschitz constant. More generally, our framework shows that supervised learning with respect to discrete distributions is not as difficult as previously thought.