Efficient Distributed Learning with Sparsity
This work addresses the challenge of efficient distributed learning for high-dimensional data, which is incremental as it builds on existing sparse learning methods.
The paper tackles the problem of distributed sparse learning in high-dimensional settings by proposing an efficient method that reduces communication rounds and computational load, achieving estimation error bounds comparable to centralized methods with constant communication rounds.
We propose a novel, efficient approach for distributed sparse learning in high-dimensions, where observations are randomly partitioned across machines. Computationally, at each round our method only requires the master machine to solve a shifted ell_1 regularized M-estimation problem, and other workers to compute the gradient. In respect of communication, the proposed approach provably matches the estimation error bound of centralized methods within constant rounds of communications (ignoring logarithmic factors). We conduct extensive experiments on both simulated and real world datasets, and demonstrate encouraging performances on high-dimensional regression and classification tasks.