Max-Diversity Distributed Learning: Theory and Algorithms
This work addresses distributed learning efficiency for machine learning practitioners, offering incremental improvements in error analysis and algorithm performance.
The paper tackles the risk performance of distributed learning for regularization empirical risk minimization, showing that greater diversity among local estimates leads to tighter risk bounds, and proposes a max-diversity algorithm (MDD) that outperforms existing divide-and-conquer methods in experiments.
We study the risk performance of distributed learning for the regularization empirical risk minimization with fast convergence rate, substantially improving the error analysis of the existing divide-and-conquer based distributed learning. An interesting theoretical finding is that the larger the diversity of each local estimate is, the tighter the risk bound is. This theoretical analysis motivates us to devise an effective maxdiversity distributed learning algorithm (MDD). Experimental results show that MDD can outperform the existing divide-andconquer methods but with a bit more time. Theoretical analysis and empirical results demonstrate that our proposed MDD is sound and effective.