Social Distancing is Good for Points too!
This work addresses a theoretical gap in a popular heuristic algorithm for data condensation, which is incremental as it modifies an existing method to provide new theoretical insights.
The paper tackled the problem of proving theoretical upper bounds for the FCNN algorithm in nearest-neighbor condensation, showing that it can behave poorly when points are too close and modifying it to enforce distance between selected points, which enabled the proof of useful upper bounds and approximation guarantees.
The nearest-neighbor rule is a well-known classification technique that, given a training set P of labeled points, classifies any unlabeled query point with the label of its closest point in P. The nearest-neighbor condensation problem aims to reduce the training set without harming the accuracy of the nearest-neighbor rule. FCNN is the most popular algorithm for condensation. It is heuristic in nature, and theoretical results for it are scarce. In this paper, we settle the question of whether reasonable upper-bounds can be proven for the size of the subset selected by FCNN. First, we show that the algorithm can behave poorly when points are too close to each other, forcing it to select many more points than necessary. We then successfully modify the algorithm to avoid such cases, thus imposing that selected points should "keep some distance". This modification is sufficient to prove useful upper-bounds, along with approximation guarantees for the algorithm.