Accelerated Proximal Stochastic Dual Coordinate Ascent for Regularized Loss Minimization
This work addresses optimization efficiency for machine learning practitioners by providing incremental improvements to existing methods for key problems.
The authors tackled the problem of regularized loss minimization by introducing a proximal version of stochastic dual coordinate ascent and accelerating it with an inner-outer iteration, achieving improved runtime rates that surpass state-of-the-art results for problems like SVM, logistic regression, ridge regression, Lasso, and multiclass SVM.
We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure. We analyze the runtime of the framework and obtain rates that improve state-of-the-art results for various key machine learning optimization problems including SVM, logistic regression, ridge regression, Lasso, and multiclass SVM. Experiments validate our theoretical findings.