Causal World Models by Unsupervised Deconfounding of Physical Dynamics
This work addresses the problem of enabling machine intelligent agents to perform counterfactual reasoning by learning world models that account for confounding factors, which is important for developing more robust and human-like AI.
This paper introduces Causal World Models (CWMs) which learn to model relationships between intervened observations and alternative futures by estimating latent confounding factors. The authors demonstrate that CWMs reduce sample complexity for reinforcement learning tasks and improve counterfactual physical reasoning.
The capability of imagining internally with a mental model of the world is vitally important for human cognition. If a machine intelligent agent can learn a world model to create a "dream" environment, it can then internally ask what-if questions -- simulate the alternative futures that haven't been experienced in the past yet -- and make optimal decisions accordingly. Existing world models are established typically by learning spatio-temporal regularities embedded from the past sensory signal without taking into account confounding factors that influence state transition dynamics. As such, they fail to answer the critical counterfactual questions about "what would have happened" if a certain action policy was taken. In this paper, we propose Causal World Models (CWMs) that allow unsupervised modeling of relationships between the intervened observations and the alternative futures by learning an estimator of the latent confounding factors. We empirically evaluate our method and demonstrate its effectiveness in a variety of physical reasoning environments. Specifically, we show reductions in sample complexity for reinforcement learning tasks and improvements in counterfactual physical reasoning.