Dual Free Adaptive Mini-batch SDCA for Empirical Risk Minimization
This is an incremental improvement for optimization in machine learning, enhancing the efficiency of SDCA methods.
The paper tackles the problem of regularized empirical risk minimization by developing a dual free mini-batch SDCA method with adaptive probabilities for non-uniform coordinate selection, showing efficiency through numerical experiments.
In this paper we develop dual free mini-batch SDCA with adaptive probabilities for regularized empirical risk minimization. This work is motivated by recent work of Shai Shalev-Shwartz on dual free SDCA method, however, we allow a non-uniform selection of "dual" coordinates in SDCA. Moreover, the probability can change over time, making it more efficient than fix uniform or non-uniform selection. We also propose an efficient procedure to generate a random non-uniform mini-batch through iterative process. The work is concluded with multiple numerical experiments to show the efficiency of proposed algorithms.