A Simple Analysis for Exp-concave Empirical Minimization with Arbitrary Convex Regularizer
This work provides a foundational analysis for optimization in machine learning, addressing a specific theoretical bottleneck in high-probability fast rates for exponential concave settings.
The paper tackles the problem of analyzing fast rates with high probability for empirical minimization in stochastic composite optimization with exponential concave loss and arbitrary convex regularization, achieving the first such result and extending existing empirical risk minimization frameworks.
In this paper, we present a simple analysis of {\bf fast rates} with {\it high probability} of {\bf empirical minimization} for {\it stochastic composite optimization} over a finite-dimensional bounded convex set with exponential concave loss functions and an arbitrary convex regularization. To the best of our knowledge, this result is the first of its kind. As a byproduct, we can directly obtain the fast rate with {\it high probability} for exponential concave empirical risk minimization with and without any convex regularization, which not only extends existing results of empirical risk minimization but also provides a unified framework for analyzing exponential concave empirical risk minimization with and without {\it any} convex regularization. Our proof is very simple only exploiting the covering number of a finite-dimensional bounded set and a concentration inequality of random vectors.