CoCoA: A General Framework for Communication-Efficient Distributed Optimization
This addresses communication bottlenecks in distributed machine learning for researchers and practitioners working with large datasets, though it appears incremental as an extension of earlier work.
The authors tackled the problem of communication inefficiency in distributed optimization for large-scale machine learning by developing CoCoA, a general framework with an efficient communication scheme that supports various regularizers including L1 and non-strongly-convex types. They demonstrated markedly improved performance over state-of-the-art methods in experiments on real distributed datasets.
The scale of modern datasets necessitates the development of efficient distributed optimization methods for machine learning. We present a general-purpose framework for distributed computing environments, CoCoA, that has an efficient communication scheme and is applicable to a wide variety of problems in machine learning and signal processing. We extend the framework to cover general non-strongly-convex regularizers, including L1-regularized problems like lasso, sparse logistic regression, and elastic net regularization, and show how earlier work can be derived as a special case. We provide convergence guarantees for the class of convex regularized loss minimization objectives, leveraging a novel approach in handling non-strongly-convex regularizers and non-smooth loss functions. The resulting framework has markedly improved performance over state-of-the-art methods, as we illustrate with an extensive set of experiments on real distributed datasets.