Exponential convergence of testing error for stochastic gradient methods
This addresses the problem of understanding convergence behavior in machine learning for researchers, providing theoretical insights into error rates, though it appears incremental as it builds on existing stochastic gradient methods.
The paper tackles the convergence rates of stochastic gradient methods for binary classification with positive definite kernels and square loss, showing that the testing error converges exponentially fast under low-noise conditions, while the excess testing loss converges slowly.
We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods. We show that while the excess testing loss (squared loss) converges slowly to zero as the number of observations (and thus iterations) goes to infinity, the testing error (classification error) converges exponentially fast if low-noise conditions are assumed.