Mohammad Mahdi Khodabandeh

Mohammad Mahdi Khodabandeh, Igor Shinkar

In this work, we continue the line of research on the complexity of distributions (Viola, Journal of Computing 2012), and study samplers defined by low degree polynomials. An $n$-tuple $P = (P_1,\dots, P_n)$ of functions $P_i \colon \mathbb{F}_2^m \to \mathbb{F}_2$ defines a distribution over $\{0,1\}^n$ in the natural way: draw $X$ uniformly at random from $\mathbb{F}_2^m$ and output $(P_1(X),\dots, P_n(X)) \in \{0,1\}^n$. We show that when $P$ is defined by polynomials of degree $d$, the total variation distance of $P$ from the product distribution $\mathrm{Ber}(1/3)^{\otimes n}$ is $1-o_n(1)$, where $o_n(1)$ is a vanishing function of $n$ for any constant degree $d$. For small values of $d$, we show the following concrete bounds. (i) For $d=1$ we have $\|P-\mathrm{Ber}(1/3)^{\otimes n}\|_{TV} \geq 1-\exp(-Ω(n))$. (ii) For $d=2$ we have $\|P-\mathrm{Ber}(1/3)^{\otimes n}\|_{TV} \geq 1-\exp(-Ω(\log(n)/\log\log(n)))$. (iii) For $d=3$ we have $\|P-\mathrm{Ber}(1/3)^{\otimes n}\|_{TV} \geq 1-\exp(-Ω(\sqrt{\log\log(n)}))$. Our results extend the recent lower bound results for sampling distributions, which have mostly focused on local samplers, small depth decision trees, and small depth circuits. As part of our proof, we establish the following result, that may be of independent interest: for any degree-$d$ polynomial $P\colon\mathbb{F}_2^m \to \mathbb{F}_2$ it holds that $\Pr_X[P(X) = 1]$ is bounded away from $1/3$ by some absolute constant $δ= δ_d>0$. Although the statement may seem obvious, we are not aware of an elementary proof of this. The proof techniques rely on the structural results for low degree polynomials, saying that any biased polynomial of degree $d$ can be written as a function of a small number of polynomials of degree $d-1$.

Tom Gur, Mohammad Mahdi Jahanara, Mohammad Mahdi Khodabandeh et al.

We continue the study of doubly-efficient proof systems for verifying agnostic PAC learning, for which we obtain the following results. - We construct an interactive protocol for learning the $t$ largest Fourier characters of a given function $f \colon \{0,1\}^n \to \{0,1\}$ up to an arbitrarily small error, wherein the verifier uses $\mathsf{poly}(t)$ random examples. This improves upon the Interactive Goldreich-Levin protocol of Goldwasser, Rothblum, Shafer, and Yehudayoff (ITCS 2021) whose sample complexity is $\mathsf{poly}(t,n)$. - For agnostically learning the class $\mathsf{AC}^0[2]$ under the uniform distribution, we build on the work of Carmosino, Impagliazzo, Kabanets, and Kolokolova (APPROX/RANDOM 2017) and design an interactive protocol, where given a function $f \colon \{0,1\}^n \to \{0,1\}$, the verifier learns the closest hypothesis up to $\mathsf{polylog}(n)$ multiplicative factor, using quasi-polynomially many random examples. In contrast, this class has been notoriously resistant even for constructing realisable learners (without a prover) using random examples. - For agnostically learning $k$-juntas under the uniform distribution, we obtain an interactive protocol, where the verifier uses $O(2^k)$ random examples to a given function $f \colon \{0,1\}^n \to \{0,1\}$. Crucially, the sample complexity of the verifier is independent of $n$. We also show that if we do not insist on doubly-efficient proof systems, then the model becomes trivial. Specifically, we show a protocol for an arbitrary class $\mathcal{C}$ of Boolean functions in the distribution-free setting, where the verifier uses $O(1)$ labeled examples to learn $f$.