Active Causal Experimentalist (ACE): Learning Intervention Strategies via Direct Preference Optimization
This addresses the challenge for experimentalists in efficiently discovering causal relationships through sequential experiments, offering a novel method that complements theoretical approaches with learned adaptation.
The paper tackles the problem of learning adaptive intervention strategies for causal discovery by proposing ACE, which uses Direct Preference Optimization to learn from pairwise comparisons, achieving a 70-71% improvement over baselines in synthetic benchmarks, physics simulations, and economic data.
Discovering causal relationships requires controlled experiments, but experimentalists face a sequential decision problem: each intervention reveals information that should inform what to try next. Traditional approaches such as random sampling, greedy information maximization, and round-robin coverage treat each decision in isolation, unable to learn adaptive strategies from experience. We propose Active Causal Experimentalist (ACE), which learns experimental design as a sequential policy. Our key insight is that while absolute information gains diminish as knowledge accumulates (making value-based RL unstable), relative comparisons between candidate interventions remain meaningful throughout. ACE exploits this via Direct Preference Optimization, learning from pairwise intervention comparisons rather than non-stationary reward magnitudes. Across synthetic benchmarks, physics simulations, and economic data, ACE achieves 70-71% improvement over baselines at equal intervention budgets (p < 0.001, Cohen's d ~ 2). Notably, the learned policy autonomously discovers that collider mechanisms require concentrated interventions on parent variables, a theoretically-grounded strategy that emerges purely from experience. This suggests preference-based learning can recover principled experimental strategies, complementing theory with learned domain adaptation.