AutoCure: Automated Tabular Data Curation Technique for ML Pipelines
This addresses the problem of time-consuming and expert-dependent data curation in domains like healthcare and finance, offering an incremental improvement in automation for ML pipelines.
The paper tackles the challenge of data preparation for machine learning by introducing AutoCure, an automated data curation pipeline for tabular data that improves model accuracy without user intervention, demonstrating superior performance against 28 traditional tool combinations.
Machine learning algorithms have become increasingly prevalent in multiple domains, such as autonomous driving, healthcare, and finance. In such domains, data preparation remains a significant challenge in developing accurate models, requiring significant expertise and time investment to search the huge search space of well-suited data curation and transformation tools. To address this challenge, we present AutoCure, a novel and configuration-free data curation pipeline that improves the quality of tabular data. Unlike traditional data curation methods, AutoCure synthetically enhances the density of the clean data fraction through an adaptive ensemble-based error detection method and a data augmentation module. In practice, AutoCure can be integrated with open source tools, e.g., Auto-sklearn, H2O, and TPOT, to promote the democratization of machine learning. As a proof of concept, we provide a comparative evaluation of AutoCure against 28 combinations of traditional data curation tools, demonstrating superior performance and predictive accuracy without user intervention. Our evaluation shows that AutoCure is an effective approach to automating data preparation and improving the accuracy of machine learning models.