Nonlinear Metric Learning for kNN and SVMs through Geometric Transformations
This work addresses the need for nonlinear metric learning in classification tasks, offering a novel approach that enhances kNN and SVM performance, though it appears incremental as it builds on existing geometric models.
The paper tackles the problem of extending linear metric learning to handle nonlinear structures by using thin-plate splines for geometric transformations, resulting in improved performance for kNN and SVM classifiers on synthetic and real-world datasets compared to state-of-the-art methods.
In recent years, research efforts to extend linear metric learning models to handle nonlinear structures have attracted great interests. In this paper, we propose a novel nonlinear solution through the utilization of deformable geometric models to learn spatially varying metrics, and apply the strategy to boost the performance of both kNN and SVM classifiers. Thin-plate splines (TPS) are chosen as the geometric model due to their remarkable versatility and representation power in accounting for high-order deformations. By transforming the input space through TPS, we can pull same-class neighbors closer while pushing different-class points farther away in kNN, as well as make the input data points more linearly separable in SVMs. Improvements in the performance of kNN classification are demonstrated through experiments on synthetic and real world datasets, with comparisons made with several state-of-the-art metric learning solutions. Our SVM-based models also achieve significant improvements over traditional linear and kernel SVMs with the same datasets.