Learning DNF through Generalized Fourier Representations
This addresses the problem of learning DNF formulas under non-uniform distributions for theoretical computer science and machine learning, representing an incremental extension of existing Fourier-based methods.
The paper tackles learning Disjunctive Normal Form (DNF) under arbitrary distributions by introducing a generalized Fourier expansion based on Bayesian networks, and shows that DNF is learnable with membership queries for a class of difference-bounded tree distributions, with bounded $L_1$ spectral norms and algorithmic extensions to unknown distributions.
The Fourier representation for the uniform distribution over the Boolean cube has found numerous applications in algorithms and complexity analysis. Notably, in learning theory, learnability of Disjunctive Normal Form (DNF) under uniform as well as product distributions has been established through such representations. This paper makes five main contributions. First, it introduces a generalized Fourier expansion that can be used with any distribution $D$ through the representation of the distribution as a Bayesian network (BN). Second, it shows that the main algorithmic tools for learning with the Fourier representation, that use membership queries to approximate functions by recovering their heavy Fourier coefficients, can be used with slight modifications with the generalized expansion. These results hold for any distribution. Third, it analyzes the $L_1$ spectral norm of conjunctions under the new expansion, showing that it is bounded for a class of distributions which can be represented by difference bounded tree BN, where a parent node in the BN representation can change the conditional expectation of a child node by at most $α<0.5$. Lower bounds are presented to show that such constraints are necessary. The fourth contribution uses these results to show the learnability of DNF with membership queries under difference bounded tree BN. The final contribution is to develop an algorithm for learning difference-bounded tree BN distributions, thus extending the DNF learnability result to cases where the distribution is not known in advance.