A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances
It addresses the problem of fragmented literature for researchers in machine learning, but it is incremental as it builds on existing surveys by offering a more unified view.
This survey tackles the challenge of providing a comprehensive overview of coreset selection methods by unifying three major research lines into a single taxonomy, highlighting overlooked subfields and offering new insights on generalization and neural scaling laws.
Coreset selection targets the challenge of finding a small, representative subset of a large dataset that preserves essential patterns for effective machine learning. Although several surveys have examined data reduction strategies before, most focus narrowly on either classical geometry-based methods or active learning techniques. In contrast, this survey presents a more comprehensive view by unifying three major lines of coreset research, namely, training-free, training-oriented, and label-free approaches, into a single taxonomy. We present subfields often overlooked by existing work, including submodular formulations, bilevel optimization, and recent progress in pseudo-labeling for unlabeled datasets. Additionally, we examine how pruning strategies influence generalization and neural scaling laws, offering new insights that are absent from prior reviews. Finally, we compare these methods under varying computational, robustness, and performance demands and highlight open challenges, such as robustness, outlier filtering, and adapting coreset selection to foundation models, for future research.