Deep Autoregressive Models as Causal Inference Engines
This work addresses the problem of scaling causal inference to complex, sequential data for researchers and practitioners in fields like AI and decision-making, representing a novel method rather than an incremental improvement.
The paper tackles the limitation of existing causal inference models in handling complex confounders and sequential actions by proposing an autoregressive framework that transforms data into sequences, enabling estimation of multiple causal quantities with a single model and improving outcome prediction accuracy in applications like maze navigation and chess endgames.
Existing causal inference (CI) models are often restricted to data with low-dimensional confounders and singleton actions. We propose an autoregressive (AR) CI framework capable of handling complex confounders and sequential actions commonly found in modern applications. Our approach accomplishes this using {\em sequencification}, which transforms data from an underlying causal diagram into a sequence of tokens. Sequencification not only accommodates training with data generated from a large class of DAGs, but also extends existing CI capabilities to estimate multiple causal quantities using a {\em single} model. We can directly compute probabilities from interventional distributions, simplifying inference and improving outcome prediction accuracy. We demonstrate that an AR model adapted for CI is efficient and effective in various complex applications such as navigating mazes, playing chess endgames, and evaluating the impact of certain keywords on paper acceptance rates, where we consider causal queries beyond standard reinforcement learning-type questions.