Sharp concentration of uniform generalization errors in binary linear classification
This work addresses theoretical guarantees for generalization errors in binary linear classification, which is incremental as it builds on existing isoperimetric arguments to provide sharper bounds.
The paper tackles the concentration of uniform generalization errors in binary linear classification by establishing Poincaré and log-Sobolev inequalities, deriving sharp concentration bounds up to moderate multiplicative constants and showing almost sure convergence in broad settings like proportionally high-dimensional regimes.
We examine the concentration of uniform generalization errors around their expectation in binary linear classification problems via an isoperimetric argument. In particular, we establish Poincaré and log-Sobolev inequalities for the joint distribution of the output labels and the label-weighted input vectors, which we apply to derive concentration bounds. The derived concentration bounds are sharp up to moderate multiplicative constants by those under well-balanced labels. In asymptotic analysis, we also show that almost sure convergence of uniform generalization errors to their expectation occurs in very broad settings, such as proportionally high-dimensional regimes. Using this convergence, we establish uniform laws of large numbers under dimension-free conditions.