Identifying Statistical Bias in Dataset Replication
This addresses bias issues in dataset replication for machine learning researchers, providing concrete recommendations to improve reliability, though it is incremental as it builds on existing replication methods.
The paper tackles the problem of statistical bias in dataset replication, showing that standard approaches skew observations; after correcting for this bias, only an estimated 3.6% ± 1.5% of an original 11.7% ± 1.0% accuracy drop in ImageNet-v2 remains unaccounted for.
Dataset replication is a useful tool for assessing whether improvements in test accuracy on a specific benchmark correspond to improvements in models' ability to generalize reliably. In this work, we present unintuitive yet significant ways in which standard approaches to dataset replication introduce statistical bias, skewing the resulting observations. We study ImageNet-v2, a replication of the ImageNet dataset on which models exhibit a significant (11-14%) drop in accuracy, even after controlling for a standard human-in-the-loop measure of data quality. We show that after correcting for the identified statistical bias, only an estimated $3.6\% \pm 1.5\%$ of the original $11.7\% \pm 1.0\%$ accuracy drop remains unaccounted for. We conclude with concrete recommendations for recognizing and avoiding bias in dataset replication. Code for our study is publicly available at http://github.com/MadryLab/dataset-replication-analysis .