Behavioral analysis of support vector machine classifier with Gaussian kernel and imbalanced data
This work addresses the computational inefficiency in SVM parameter tuning for researchers and practitioners dealing with classification tasks, but it is incremental as it builds on existing SVM methods.
The authors tackled the problem of tuning SVM parameters efficiently by analyzing the behavior of SVM with Gaussian and linear kernels on balanced and imbalanced data, and proposed a novel search algorithm that reduces computational time by searching in two one-dimensional spaces instead of one two-dimensional space, achieving faster and more effective results than other strategies.
The parameters of support vector machines (SVMs) such as the penalty parameter and the kernel parameters have a great impact on the classification accuracy and the complexity of the SVM model. Therefore, the model selection in SVM involves the tuning of these parameters. However, these parameters are usually tuned and used as a black box, without understanding the mathematical background or internal details. In this paper, the behavior of the SVM classification model is analyzed when these parameters take different values with balanced and imbalanced data. This analysis including visualization, mathematical and geometrical interpretations and illustrative numerical examples with the aim of providing the basics of the Gaussian and linear kernel functions with SVM. From this analysis, we proposed a novel search algorithm. In this algorithm, we search for the optimal SVM parameters into two one-dimensional spaces instead of searching into one two-dimensional space. This reduces the computational time significantly. Moreover, in our algorithm, from the analysis of the data, the range of kernel function can be expected. This also reduces the search space and hence reduces the required computational time. Different experiments were conducted to evaluate our search algorithm using different balanced and imbalanced datasets. The results demonstrated how the proposed strategy is fast and effective than other searching strategies.