A metric learning perspective of SVM: on the relation of SVM and LMNN
This work provides a theoretical unification of two popular machine learning algorithms, offering insights for researchers in metric learning and classification, though it is incremental in nature.
The paper unifies Support Vector Machines (SVM) and Large Margin Nearest Neighbor (LMNN) by analyzing SVM from a metric learning perspective, showing LMNN as local SVM-like models, and introduces epsilon-SVM, which performs favorably on benchmark datasets compared to both LMNN and SVM.
Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVM s we derive a novel variant of SVMs, epsilon-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that epsilon-SVM compares favorably both with respect to LMNN and SVM.