Robust agents learn causal world models
This provides a theoretical foundation for causal reasoning in AI, impacting fields like transfer learning and causal inference, though it is incremental in linking existing concepts.
The paper tackles the problem of whether agents need causal models to generalize robustly across domains, showing that agents achieving a regret bound under distributional shifts must have learned an approximate causal model that converges to the true one for optimal agents.
It has long been hypothesised that causal reasoning plays a fundamental role in robust and general intelligence. However, it is not known if agents must learn causal models in order to generalise to new domains, or if other inductive biases are sufficient. We answer this question, showing that any agent capable of satisfying a regret bound under a large set of distributional shifts must have learned an approximate causal model of the data generating process, which converges to the true causal model for optimal agents. We discuss the implications of this result for several research areas including transfer learning and causal inference.