F-SVM: Combination of Feature Transformation and SVM Learning via Convex Relaxation
This work offers an incremental improvement for machine learning practitioners by providing a more efficient and effective method for SVM-based classification tasks.
The authors tackled the problem of improving SVM generalization by jointly learning feature transformation and classifier, addressing limitations of prior radius-margin methods that were non-convex or restrictive. They proposed F-SVM, a convex model that outperformed standard SVM and existing radius-margin SVMs on UCI and LFW datasets.
The generalization error bound of support vector machine (SVM) depends on the ratio of radius and margin, while standard SVM only considers the maximization of the margin but ignores the minimization of the radius. Several approaches have been proposed to integrate radius and margin for joint learning of feature transformation and SVM classifier. However, most of them either require the form of the transformation matrix to be diagonal, or are non-convex and computationally expensive. In this paper, we suggest a novel approximation for the radius of minimum enclosing ball (MEB) in feature space, and then propose a convex radius-margin based SVM model for joint learning of feature transformation and SVM classifier, i.e., F-SVM. An alternating minimization method is adopted to solve the F-SVM model, where the feature transformation is updatedvia gradient descent and the classifier is updated by employing the existing SVM solver. By incorporating with kernel principal component analysis, F-SVM is further extended for joint learning of nonlinear transformation and classifier. Experimental results on the UCI machine learning datasets and the LFW face datasets show that F-SVM outperforms the standard SVM and the existing radius-margin based SVMs, e.g., RMM, R-SVM+ and R-SVM+μ.