Private PAC learning implies finite Littlestone dimension
This work addresses the fundamental problem of understanding the sample complexity of private learning for researchers in differential privacy and computational learning theory, providing a significant theoretical advance.
The paper proves that any approximately differentially private learning algorithm for a class with Littlestone dimension d requires at least Ω(log*(d)) examples, and as a corollary, it resolves an open question by showing that the class of thresholds over natural numbers cannot be learned privately.
We show that every approximately differentially private learning algorithm (possibly improper) for a class $H$ with Littlestone dimension~$d$ requires $Ω\bigl(\log^*(d)\bigr)$ examples. As a corollary it follows that the class of thresholds over $\mathbb{N}$ can not be learned in a private manner; this resolves open question due to [Bun et al., 2015, Feldman and Xiao, 2015]. We leave as an open question whether every class with a finite Littlestone dimension can be learned by an approximately differentially private algorithm.