VDSAgents: A PCS-Guided Multi-Agent System for Veridical Data Science Automation
This addresses trustworthiness and robustness issues in automated data science for users dealing with noisy real-world data, representing a novel integration of principles rather than an incremental improvement.
The paper tackles the problem of LLM-driven data science systems lacking scientific guidance by introducing VDSAgents, a multi-agent system based on PCS principles, which outperforms state-of-the-art systems like AutoKaggle and DataInterpreter on nine diverse datasets.
Large language models (LLMs) become increasingly integrated into data science workflows for automated system design. However, these LLM-driven data science systems rely solely on the internal reasoning of LLMs, lacking guidance from scientific and theoretical principles. This limits their trustworthiness and robustness, especially when dealing with noisy and complex real-world datasets. This paper provides VDSAgents, a multi-agent system grounded in the Predictability-Computability-Stability (PCS) principles proposed in the Veridical Data Science (VDS) framework. Guided by PCS principles, the system implements a modular workflow for data cleaning, feature engineering, modeling, and evaluation. Each phase is handled by an elegant agent, incorporating perturbation analysis, unit testing, and model validation to ensure both functionality and scientific auditability. We evaluate VDSAgents on nine datasets with diverse characteristics, comparing it with state-of-the-art end-to-end data science systems, such as AutoKaggle and DataInterpreter, using DeepSeek-V3 and GPT-4o as backends. VDSAgents consistently outperforms the results of AutoKaggle and DataInterpreter, which validates the feasibility of embedding PCS principles into LLM-driven data science automation.