Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machine
This work addresses the problem of efficient distributed classification for high-dimensional data in decentralized networks, representing an incremental improvement over existing methods.
The paper tackled the challenge of slow convergence in decentralized learning for high-dimensional classification by developing a convolution-based smoothing technique for the hinge loss and a generalized ADMM algorithm, achieving provable linear convergence and near-optimal statistical performance.
This paper addresses the problem of efficiently classifying high-dimensional data over decentralized networks. Penalized support vector machines (SVMs) are widely used for high-dimensional classification tasks. However, the double nonsmoothness of the objective function poses significant challenges in developing efficient decentralized learning methods. Many existing procedures suffer from slow, sublinear convergence rates. To overcome this limitation, we consider a convolution-based smoothing technique for the nonsmooth hinge loss function. The resulting loss function remains convex and smooth. We then develop an efficient generalized alternating direction method of multipliers (ADMM) algorithm for solving penalized SVM over decentralized networks. Our theoretical contributions are twofold. First, we establish that our generalized ADMM algorithm achieves provable linear convergence with a simple implementation. Second, after a sufficient number of ADMM iterations, the final sparse estimator attains near-optimal statistical convergence and accurately recovers the true support of the underlying parameters. Extensive numerical experiments on both simulated and real-world datasets validate our theoretical findings.