Faster SVM Training via Conjugate SMO
This work addresses the efficiency of SVM training, particularly beneficial for users performing hyper-parameter tuning via grid-search, though it is incremental as it builds on the existing SMO algorithm.
The paper tackles the problem of slow SVM training by proposing an improved SMO algorithm using Conjugate Descent, which reduces the number of iterations needed for convergence while only modestly increasing per-iteration cost, as shown through implementation in LIBSVM and experimental results indicating faster training for many hyper-parameter configurations.
We propose an improved version of the SMO algorithm for training classification and regression SVMs, based on a Conjugate Descent procedure. This new approach only involves a modest increase on the computational cost of each iteration but, in turn, usually results in a substantial decrease in the number of iterations required to converge to a given precision. Besides, we prove convergence of the iterates of this new Conjugate SMO as well as a linear rate when the kernel matrix is positive definite. We have implemented Conjugate SMO within the LIBSVM library and show experimentally that it is faster for many hyper-parameter configurations, being often a better option than second order SMO when performing a grid-search for SVM tuning.