Two-Stage Metric Learning
This work addresses the challenge of learning effective distance metrics in machine learning, particularly for applications requiring non-positive semi-definite similarity measures, but it appears incremental as it builds on existing metric learning concepts.
The paper tackles the problem of metric learning by introducing a two-stage algorithm that maps instances to probability distributions and uses Fisher information distance, resulting in significant performance improvements over other metric learning methods and SVM on multiple datasets.
In this paper, we present a novel two-stage metric learning algorithm. We first map each learning instance to a probability distribution by computing its similarities to a set of fixed anchor points. Then, we define the distance in the input data space as the Fisher information distance on the associated statistical manifold. This induces in the input data space a new family of distance metric with unique properties. Unlike kernelized metric learning, we do not require the similarity measure to be positive semi-definite. Moreover, it can also be interpreted as a local metric learning algorithm with well defined distance approximation. We evaluate its performance on a number of datasets. It outperforms significantly other metric learning methods and SVM.