Fast Distributed Coordinate Descent for Non-Strongly Convex Losses
This work addresses efficient large-scale optimization for distributed systems, offering incremental improvements in convergence rates for non-strongly convex functions.
The authors tackled the problem of minimizing regularized non-strongly convex loss functions in a distributed setting, achieving an optimal O(1/k^2) convergence rate and demonstrating the method's capability by solving a LASSO problem with 50 billion variables on a supercomputer.
We propose an efficient distributed randomized coordinate descent method for minimizing regularized non-strongly convex loss functions. The method attains the optimal $O(1/k^2)$ convergence rate, where $k$ is the iteration counter. The core of the work is the theoretical study of stepsize parameters. We have implemented the method on Archer - the largest supercomputer in the UK - and show that the method is capable of solving a (synthetic) LASSO optimization problem with 50 billion variables.