Weighted Distance Nearest Neighbor Condensing
This work addresses a long-standing problem in machine learning for improving nearest neighbor classification efficiency and accuracy, though it appears incremental as it builds on existing condensing methods.
The paper tackles the problem of nearest neighbor condensing by introducing weighted distances, where points in the condensed set have weights, and new points are labeled based on weighted nearest neighbors. It shows that this approach can produce dramatically better condensing than standard methods while maintaining similar generalization bounds, with promising empirical results.
The problem of nearest neighbor condensing has enjoyed a long history of study, both in its theoretical and practical aspects. In this paper, we introduce the problem of weighted distance nearest neighbor condensing, where one assigns weights to each point of the condensed set, and then new points are labeled based on their weighted distance nearest neighbor in the condensed set. We study the theoretical properties of this new model, and show that it can produce dramatically better condensing than the standard nearest neighbor rule, yet is characterized by generalization bounds almost identical to the latter. We then suggest a condensing heuristic for our new problem. We demonstrate Bayes consistency for this heuristic, and also show promising empirical results.