Hardness of High-Dimensional Linear Classification
This work addresses a fundamental problem in computational geometry and machine learning by providing rigorous hardness results for high-dimensional linear classification, which is incremental in closing theoretical gaps but foundational for algorithm design.
The paper tackles the problem of establishing lower bounds for the Maximum Halfspace Discrepancy problem in high-dimensional linear classification, closing the gap between known upper bounds and polynomial lower bounds by proving exponential-in-dimension lower bounds up to polylogarithmic terms, with results including matching lower bounds of Ω̃(n^d) and Ω̃(1/ε^d) based on Affine Degeneracy testing.
We establish new exponential in dimension lower bounds for the Maximum Halfspace Discrepancy problem, which models linear classification. Both are fundamental problems in computational geometry and machine learning in their exact and approximate forms. However, only $O(n^d)$ and respectively $\tilde O(1/\varepsilon^d)$ upper bounds are known and complemented by polynomial lower bounds that do not support the exponential in dimension dependence. We close this gap up to polylogarithmic terms by reduction from widely-believed hardness conjectures for Affine Degeneracy testing and $k$-Sum problems. Our reductions yield matching lower bounds of $\tildeΩ(n^d)$ and respectively $\tildeΩ(1/\varepsilon^d)$ based on Affine Degeneracy testing, and $\tildeΩ(n^{d/2})$ and respectively $\tildeΩ(1/\varepsilon^{d/2})$ conditioned on $k$-Sum. The first bound also holds unconditionally if the computational model is restricted to make sidedness queries, which corresponds to a widely spread setting implemented and optimized in many contemporary algorithms and computing paradigms.