Radial basis function kernel optimization for Support Vector Machine classifiers
This work addresses a specific optimization issue for SVM users, but it is incremental as it builds on existing SVM and gradient descent methods.
The authors tackled the problem of the Radial Basis Function kernel's dependence on initial hyperparameter values in Support Vector Machines by proposing OKSVM, an algorithm that automatically learns the hyperparameter and adjusts SVM weights simultaneously, resulting in better performance irrespective of initial values.
Support Vector Machines (SVMs) are still one of the most popular and precise classifiers. The Radial Basis Function (RBF) kernel has been used in SVMs to separate among classes with considerable success. However, there is an intrinsic dependence on the initial value of the kernel hyperparameter. In this work, we propose OKSVM, an algorithm that automatically learns the RBF kernel hyperparameter and adjusts the SVM weights simultaneously. The proposed optimization technique is based on a gradient descent method. We analyze the performance of our approach with respect to the classical SVM for classification on synthetic and real data. Experimental results show that OKSVM performs better irrespective of the initial values of the RBF hyperparameter.