Jan Vondrák

Vitaly Feldman, Jan Vondrak

Submodular and fractionally subadditive (or equivalently XOS) functions play a fundamental role in combinatorial optimization, algorithmic game theory and machine learning. Motivated by learnability of these classes of functions from random examples, we consider the question of how well such functions can be approximated by low-degree polynomials in $\ell_2$ norm over the uniform distribution. This question is equivalent to understanding of the concentration of Fourier weight on low-degree coefficients, a central concept in Fourier analysis. We show that 1. For any submodular function $f:\{0,1\}^n \rightarrow [0,1]$, there is a polynomial of degree $O(\log (1/ε) / ε^{4/5})$ approximating $f$ within $ε$ in $\ell_2$, and there is a submodular function that requires degree $Ω(1/ε^{4/5})$. 2. For any XOS function $f:\{0,1\}^n \rightarrow [0,1]$, there is a polynomial of degree $O(1/ε)$ and there exists an XOS function that requires degree $Ω(1/ε)$. This improves on previous approaches that all showed an upper bound of $O(1/ε^2)$ for submodular and XOS functions. The best previous lower bound was $Ω(1/ε^{2/3})$ for monotone submodular functions. Our techniques reveal new structural properties of submodular and XOS functions and the upper bounds lead to nearly optimal PAC learning algorithms for these classes of functions.

Vitaly Feldman, Jan Vondrak

We investigate the approximability of several classes of real-valued functions by functions of a small number of variables ({\em juntas}). Our main results are tight bounds on the number of variables required to approximate a function $f:\{0,1\}^n \rightarrow [0,1]$ within $\ell_2$-error $ε$ over the uniform distribution: 1. If $f$ is submodular, then it is $ε$-close to a function of $O(\frac{1}{ε^2} \log \frac{1}ε)$ variables. This is an exponential improvement over previously known results. We note that $Ω(\frac{1}{ε^2})$ variables are necessary even for linear functions. 2. If $f$ is fractionally subadditive (XOS) it is $ε$-close to a function of $2^{O(1/ε^2)}$ variables. This result holds for all functions with low total $\ell_1$-influence and is a real-valued analogue of Friedgut's theorem for boolean functions. We show that $2^{Ω(1/ε)}$ variables are necessary even for XOS functions. As applications of these results, we provide learning algorithms over the uniform distribution. For XOS functions, we give a PAC learning algorithm that runs in time $2^{poly(1/ε)} poly(n)$. For submodular functions we give an algorithm in the more demanding PMAC learning model (Balcan and Harvey, 2011) which requires a multiplicative $1+γ$ factor approximation with probability at least $1-ε$ over the target distribution. Our uniform distribution algorithm runs in time $2^{poly(1/(γε))} poly(n)$. This is the first algorithm in the PMAC model that over the uniform distribution can achieve a constant approximation factor arbitrarily close to 1 for all submodular functions. As follows from the lower bounds in (Feldman et al., 2013) both of these algorithms are close to optimal. We also give applications for proper learning, testing and agnostic learning with value queries of these classes.