A New Lower Bound for Agnostic Learning with Sample Compression Schemes
This provides a tight characterization for theoretical machine learning, addressing fundamental limits in agnostic learning, though it is incremental in refining existing bounds.
The paper tackles the problem of determining worst-case rates for excess risk in agnostic learning with sample compression schemes, finding that optimal convergence rates are of the form √(k log(n/k)/n), contrasting with √(k/n) for VC dimension classes.
We establish a tight characterization of the worst-case rates for the excess risk of agnostic learning with sample compression schemes and for uniform convergence for agnostic sample compression schemes. In particular, we find that the optimal rates of convergence for size-$k$ agnostic sample compression schemes are of the form $\sqrt{\frac{k \log(n/k)}{n}}$, which contrasts with agnostic learning with classes of VC dimension $k$, where the optimal rates are of the form $\sqrt{\frac{k}{n}}$.