Estimating long-term causal effects from short-term experiments and long-term observational data with unobserved confounding
This addresses a critical challenge in causal inference for domains like healthcare, where long-term randomized trials are impractical, but it is incremental as it builds on prior work by handling unobserved confounders.
The paper tackles the problem of estimating long-term causal effects when only short-term experimental data and long-term observational data with unobserved confounders are available, by developing an unbiased estimator that combines regression residuals with short-term outcomes to create an instrumental variable, achieving accurate results as validated on synthetic and real-world stroke trial data.
Understanding and quantifying cause and effect is an important problem in many domains. The generally-agreed solution to this problem is to perform a randomised controlled trial. However, even when randomised controlled trials can be performed, they usually have relatively short duration's due to cost considerations. This makes learning long-term causal effects a very challenging task in practice, since the long-term outcome is only observed after a long delay. In this paper, we study the identification and estimation of long-term treatment effects when both experimental and observational data are available. Previous work provided an estimation strategy to determine long-term causal effects from such data regimes. However, this strategy only works if one assumes there are no unobserved confounders in the observational data. In this paper, we specifically address the challenging case where unmeasured confounders are present in the observational data. Our long-term causal effect estimator is obtained by combining regression residuals with short-term experimental outcomes in a specific manner to create an instrumental variable, which is then used to quantify the long-term causal effect through instrumental variable regression. We prove this estimator is unbiased, and analytically study its variance. In the context of the front-door causal structure, this provides a new causal estimator, which may be of independent interest. Finally, we empirically test our approach on synthetic-data, as well as real-data from the International Stroke Trial.