Distributed Mini-Batch SDCA
This work provides an incremental improvement in optimization theory for machine learning practitioners working with distributed or large-scale datasets.
The paper tackles the problem of improving the analysis of mini-batched stochastic dual coordinate ascent for regularized empirical loss minimization, such as SVM objectives, by allowing flexible sampling schemes including distributed data and incorporating dependencies on loss smoothness and data spread.
We present an improved analysis of mini-batched stochastic dual coordinate ascent for regularized empirical loss minimization (i.e. SVM and SVM-type objectives). Our analysis allows for flexible sampling schemes, including where data is distribute across machines, and combines a dependence on the smoothness of the loss and/or the data spread (measured through the spectral norm).