Fix your Models by Fixing your Datasets
This work addresses the data tooling gap for ML teams, enabling more intelligent data discovery and pruning to build performant models with wider generalizability, though it is incremental as it builds on existing data-centric techniques.
The paper tackles the lack of streamlined tools for improving training data quality in machine learning by introducing a systematic framework to identify noisy or mislabeled samples and the most informative samples for model performance, demonstrating its efficacy on public and private enterprise datasets from Fortune 500 companies.
The quality of underlying training data is very crucial for building performant machine learning models with wider generalizabilty. However, current machine learning (ML) tools lack streamlined processes for improving the data quality. So, getting data quality insights and iteratively pruning the errors to obtain a dataset which is most representative of downstream use cases is still an ad-hoc manual process. Our work addresses this data tooling gap, required to build improved ML workflows purely through data-centric techniques. More specifically, we introduce a systematic framework for (1) finding noisy or mislabelled samples in the dataset and, (2) identifying the most informative samples, which when included in training would provide maximal model performance lift. We demonstrate the efficacy of our framework on public as well as private enterprise datasets of two Fortune 500 companies, and are confident this work will form the basis for ML teams to perform more intelligent data discovery and pruning.