Bayesian Neighbourhood Component Analysis
This work addresses metric learning for KNN classifiers, particularly in scenarios with limited or noisy data, though it is incremental as it builds on existing Neighbourhood Component Analysis.
The authors tackled the problem of learning robust distance metrics from small or noisy datasets by introducing Bayesian Neighbourhood Component Analysis, which outperformed a previous Bayesian metric learning method on several public datasets.
Learning a good distance metric in feature space potentially improves the performance of the KNN classifier and is useful in many real-world applications. Many metric learning algorithms are however based on the point estimation of a quadratic optimization problem, which is time-consuming, susceptible to overfitting, and lack a natural mechanism to reason with parameter uncertainty, an important property useful especially when the training set is small and/or noisy. To deal with these issues, we present a novel Bayesian metric learning method, called Bayesian NCA, based on the well-known Neighbourhood Component Analysis method, in which the metric posterior is characterized by the local label consistency constraints of observations, encoded with a similarity graph instead of independent pairwise constraints. For efficient Bayesian optimization, we explore the variational lower bound over the log-likelihood of the original NCA objective. Experiments on several publicly available datasets demonstrate that the proposed method is able to learn robust metric measures from small size dataset and/or from challenging training set with labels contaminated by errors. The proposed method is also shown to outperform a previous pairwise constrained Bayesian metric learning method.