ORCA: ORchestrating Causal Agent
This addresses the problem of managing dependencies and consistency in causal analysis for users working with complex databases, representing a novel method for a known bottleneck.
The paper tackles the challenge of conducting coherent causal analysis on relational databases by proposing ORCA, an interactive multi-agent framework that maintains shared state and includes human checkpoints. In a user study, ORCA increased successful end-to-end analysis completion by 42 percentage points, reduced ATE error, and cut time spent on repetitive tasks by 76% on average.
Causal analysis on relational databases is challenging, as analysis datasets must be repeatedly queried from complex schemas. Recent LLM systems can automate individual steps, but they hardly manage dependencies across analysis stages, making it difficult to preserve consistency between causal hypothesis. We propose ORCA (ORchestrating Causal Agent), an interactive multi-agent framework to enable coherent causal analysis on relational databases by maintaining shared state and introducing human checkpoints. In a controlled user study, participants using ORCA successfully completed end-to-end analysis more often than with a baseline LLM (GPT-4o-mini) assistant by 42 percentage points, achieved substantially lower ATE error, and reduced time spent on repetitive data exploration and query refinement by 76\% on average. These results show that ORCA improves both how users interact with the causal analysis pipeline and the reliability of the resulting causal conclusions.