Autodata: An agentic data scientist to create high quality synthetic data
For AI practitioners, this provides a scalable method to generate synthetic data that improves model performance, potentially shifting how training data is produced.
Autodata introduces an agentic data scientist that creates high-quality synthetic training and evaluation data, outperforming classical methods on computer science, legal reasoning, and mathematical reasoning tasks. Meta-optimizing the agent yields further improvements, converting inference compute into better model training.
We introduce Autodata, a general method that enables AI agents to act as data scientists who build high quality training and evaluation data. We show how to train (meta-optimize) such a data scientist agent, so that it learns to create even stronger data. We describe the overall formulation, and a specific practical implementation, Agentic Self-Instruct. We conduct experiments on computer science research tasks, legal reasoning tasks and reasoning with mathematical objects, where we obtain improved results compared to classical synthetic dataset creation methods. Further, meta-optimizing the data scientist agent itself delivers an even larger performance uplift. Agentic data creation provides a way to convert increased inference compute into higher quality model training. Overall, we believe this direction has the potential to change the way we build AI data.