GADGET SVM: A Gossip-bAseD sub-GradiEnT Solver for Linear SVMs
This addresses the need for scalable SVM algorithms for machine learning researchers dealing with big data, though it is incremental as it builds on existing distributed and primal formulation approaches.
The paper tackles the scalability problem of training linear SVMs on large datasets by proposing GADGET SVM, a distributed gossip-based sub-gradient solver, achieving performance comparable to centralized and online methods.
In the era of big data, an important weapon in a machine learning researcher's arsenal is a scalable Support Vector Machine (SVM) algorithm. SVMs are extensively used for solving classification problems. Traditional algorithms for learning SVMs often scale super linearly with training set size which becomes infeasible very quickly for large data sets. In recent years, scalable algorithms have been designed which study the primal or dual formulations of the problem. This often suggests a way to decompose the problem and facilitate development of distributed algorithms. In this paper, we present a distributed algorithm for learning linear Support Vector Machines in the primal form for binary classification called Gossip-bAseD sub-GradiEnT (GADGET) SVM. The algorithm is designed such that it can be executed locally on nodes of a distributed system. Each node processes its local homogeneously partitioned data and learns a primal SVM model. It then gossips with random neighbors about the classifier learnt and uses this information to update the model. Extensive theoretical and empirical results suggest that this anytime algorithm has performance comparable to its centralized and online counterparts.