Majorization-Minimization for sparse SVMs
This work addresses the challenge of efficient and sparse SVM training for binary classification tasks, offering incremental improvements in computational speed and feature selection.
The authors tackled the problem of training sparse Support Vector Machines (SVMs) by proposing a smooth sparse-promoting-regularized squared hinge loss minimization, which enables the use of majorization-minimization methods for quick training and enhances feature selection. Numerical tests on three datasets showed good performance in accuracy, precision, recall, F1 score, and computational cost, though specific numbers were not provided.
Several decades ago, Support Vector Machines (SVMs) were introduced for performing binary classification tasks, under a supervised framework. Nowadays, they often outperform other supervised methods and remain one of the most popular approaches in the machine learning arena. In this work, we investigate the training of SVMs through a smooth sparse-promoting-regularized squared hinge loss minimization. This choice paves the way to the application of quick training methods built on majorization-minimization approaches, benefiting from the Lipschitz differentiabililty of the loss function. Moreover, the proposed approach allows us to handle sparsity-preserving regularizers promoting the selection of the most significant features, so enhancing the performance. Numerical tests and comparisons conducted on three different datasets demonstrate the good performance of the proposed methodology in terms of qualitative metrics (accuracy, precision, recall, and F 1 score) as well as computational cost.