A Quasi-Newton Method for Large Scale Support Vector Machines
This work provides an incremental improvement for machine learning practitioners needing efficient optimization in high-dimensional SVM tasks.
The paper tackles the problem of solving large-scale support vector machine classification by adapting a regularized stochastic BFGS quasi-Newton method, achieving almost sure convergence to the optimal classifier with a linear expected rate and smooth degradation with feature dimensionality.
This paper adapts a recently developed regularized stochastic version of the Broyden, Fletcher, Goldfarb, and Shanno (BFGS) quasi-Newton method for the solution of support vector machine classification problems. The proposed method is shown to converge almost surely to the optimal classifier at a rate that is linear in expectation. Numerical results show that the proposed method exhibits a convergence rate that degrades smoothly with the dimensionality of the feature vectors.