TEA-Time: Transporting Effects Across Time
This work is significant for researchers and practitioners who need to apply treatment effect estimates from past experiments to current or future time periods, addressing the challenge of temporal generalizability.
This paper addresses the problem of extrapolating treatment effects from randomized controlled trials to different time periods. It proposes a framework where the transported average treatment effect (TATE) decomposes into an observed average treatment effect and a temporal ratio, and provides two identification strategies with doubly robust, semiparametrically efficient estimators. Monte Carlo simulations confirm nominal coverage, and application to A/B tests shows a variance-bias tradeoff between the two strategies.
Treatment effects estimated from randomized controlled trials are local not only to the study population but also to the time at which the trial was conducted. We develop a framework for temporal transportation: extrapolating treatment effects to time periods where no experiment was conducted. We target the transported average treatment effect (TATE) and show that under a separable temporal effects assumption, the TATE decomposes into an observed average treatment effect and a temporal ratio. We provide two identification strategies -- one using replicated trials comparing the same treatments at different times, another using common treatment arms observed across time -- and develop doubly robust, semiparametrically efficient estimators for each. Monte Carlo simulations confirm that both estimators achieve nominal coverage, with the common arm strategy yielding substantial efficiency gains when its stronger assumptions hold. We apply our methods to A/B tests from the Upworthy Research Archive, demonstrating that the two strategies exhibit a variance-bias tradeoff: the common arm approach offers greater precision but may incur bias when treatments interact heterogeneously with temporal factors.