The Polynomial Method is Universal for Distribution-Free Correlational SQ Learning
This work provides a foundational insight for theoretical machine learning researchers by establishing a universal approach for CSQ learning, though it is incremental as it builds on prior results.
The paper tackles the problem of distribution-free learning for Boolean function classes by proving that lower bounds on threshold or approximate degree directly imply correlational statistical query (CSQ) lower bounds for PAC or agnostic learning, showing that the polynomial method is universal and best-possible for this setting.
We consider the problem of distribution-free learning for Boolean function classes in the PAC and agnostic models. Generalizing a beautiful work of Malach and Shalev-Shwartz (2022) that gave tight correlational SQ (CSQ) lower bounds for learning DNF formulas, we give new proofs that lower bounds on the threshold or approximate degree of any function class directly imply CSQ lower bounds for PAC or agnostic learning respectively. While such bounds implicitly follow by combining prior results by Feldman (2008, 2012) and Sherstov (2008, 2011), to our knowledge the precise statements we give had not appeared in this form before. Moreover, our proofs are simple and largely self-contained. These lower bounds match corresponding positive results using upper bounds on the threshold or approximate degree in the SQ model for PAC or agnostic learning, and in this sense these results show that the polynomial method is a universal, best-possible approach for distribution-free CSQ learning.