Recipe for Fast Large-scale SVM Training: Polishing, Parallelism, and more RAM!
This work addresses the scalability issue for SVM training in machine learning, particularly for tabular data, though it is incremental as it builds on existing techniques.
The authors tackled the problem of long training times for non-linear kernel SVMs on large datasets by combining approximate solvers and parallel GPU implementations, achieving training of a large-margin classifier on ImageNet in 24 minutes.
Support vector machines (SVMs) are a standard method in the machine learning toolbox, in particular for tabular data. Non-linear kernel SVMs often deliver highly accurate predictors, however, at the cost of long training times. That problem is aggravated by the exponential growth of data volumes over time. It was tackled in the past mainly by two types of techniques: approximate solvers, and parallel GPU implementations. In this work, we combine both approaches to design an extremely fast dual SVM solver. We fully exploit the capabilities of modern compute servers: many-core architectures, multiple high-end GPUs, and large random access memory. On such a machine, we train a large-margin classifier on the ImageNet data set in 24 minutes.