A PTAS for Agnostically Learning Halfspaces
This provides a near-optimal solution for learning halfspaces in agnostic settings, which is important for machine learning practitioners dealing with noisy data, though it is incremental as it builds on existing techniques.
The paper tackles the problem of agnostically learning halfspaces under the uniform distribution on a sphere, presenting a polynomial-time approximation scheme (PTAS) that returns a classifier with error at most (1+μ)opt + ε, improving upon prior work with an unspecified constant approximation ratio.
We present a PTAS for agnostically learning halfspaces w.r.t. the uniform distribution on the $d$ dimensional sphere. Namely, we show that for every $μ>0$ there is an algorithm that runs in time $\mathrm{poly}(d,\frac{1}ε)$, and is guaranteed to return a classifier with error at most $(1+μ)\mathrm{opt}+ε$, where $\mathrm{opt}$ is the error of the best halfspace classifier. This improves on Awasthi, Balcan and Long [ABL14] who showed an algorithm with an (unspecified) constant approximation ratio. Our algorithm combines the classical technique of polynomial regression (e.g. [LMN89, KKMS05]), together with the new localization technique of [ABL14].