Scaling Up Coordinate Descent Algorithms for Large $\ell_1$ Regularization Problems
This work addresses the problem of efficient large-scale optimization for machine learning practitioners, but it appears incremental as it builds on existing parallel methods.
The authors tackled the challenge of scaling coordinate descent algorithms for large-scale ℓ1 regularization problems by introducing a generic parallel framework and two novel algorithms, Thread-Greedy CD and Coloring-Based CD, with performance measurements provided for an OpenMP implementation.
We present a generic framework for parallel coordinate descent (CD) algorithms that includes, as special cases, the original sequential algorithms Cyclic CD and Stochastic CD, as well as the recent parallel Shotgun algorithm. We introduce two novel parallel algorithms that are also special cases---Thread-Greedy CD and Coloring-Based CD---and give performance measurements for an OpenMP implementation of these.