Towards an Analytical Definition of Sufficient Data
This work addresses data efficiency in machine learning by identifying redundant samples, but it is incremental as it builds on existing centroid-based analysis without introducing a new paradigm.
The paper tackles the problem of identifying which training samples are most informative for classification by analyzing their positions relative to class centroids in reduced dimensional space, showing that excluding up to 2% of data near centroids does not significantly affect performance across five datasets.
We show that, for each of five datasets of increasing complexity, certain training samples are more informative of class membership than others. These samples can be identified a priori to training by analyzing their position in reduced dimensional space relative to the classes' centroids. Specifically, we demonstrate that samples nearer the classes' centroids are less informative than those that are furthest from it. For all five datasets, we show that there is no statistically significant difference between training on the entire training set and when excluding up to 2% of the data nearest to each class's centroid.