Learning $\mathsf{AC}^0$ Under Graphical Models
This addresses a longstanding challenge in computational learning theory by moving beyond unrealistic product structures to more natural correlated distributions, though it is incremental as it builds on prior work like the low-degree algorithm.
The paper tackles the problem of learning constant-depth circuits (AC^0) under correlated distributions, specifically graphical models with polynomial growth and strong spatial mixing, by developing quasipolynomial-time algorithms that extend beyond the product setting.
In a landmark result, Linial, Mansour and Nisan (J. ACM 1993) gave a quasipolynomial-time algorithm for learning constant-depth circuits given labeled i.i.d. samples under the uniform distribution. Their work has had a deep and lasting legacy in computational learning theory, in particular introducing the $\textit{low-degree algorithm}$. However, an important critique of many results and techniques in the area is the reliance on product structure, which is unlikely to hold in realistic settings. Obtaining similar learning guarantees for more natural correlated distributions has been a longstanding challenge in the field. In particular, we give quasipolynomial-time algorithms for learning $\mathsf{AC}^0$ substantially beyond the product setting, when the inputs come from any graphical model with polynomial growth that exhibits strong spatial mixing. The main technical challenge is in giving a workaround to Fourier analysis, which we do by showing how new sampling algorithms allow us to transfer statements about low-degree polynomial approximation under the uniform setting to graphical models. Our approach is general enough to extend to other well-studied function classes, like monotone functions and halfspaces.