A Coreset Learning Reality Check
This work addresses the problem of evaluating subsampling algorithms for researchers and practitioners, highlighting that many existing methods are incremental and may not deliver promised gains.
The paper conducted a comprehensive evaluation of subsampling methods for logistic regression, revealing that many proposed methods often fail to outperform simple uniform subsampling, with inconsistencies in their effectiveness.
Subsampling algorithms are a natural approach to reduce data size before fitting models on massive datasets. In recent years, several works have proposed methods for subsampling rows from a data matrix while maintaining relevant information for classification. While these works are supported by theory and limited experiments, to date there has not been a comprehensive evaluation of these methods. In our work, we directly compare multiple methods for logistic regression drawn from the coreset and optimal subsampling literature and discover inconsistencies in their effectiveness. In many cases, methods do not outperform simple uniform subsampling.