A Reinforcement Learning Approach to Estimating Long-term Treatment Effects
This work addresses the challenge of measuring long-term effects in business and healthcare applications, offering a method to reduce the time and cost of experiments, though it appears incremental as it builds on existing RL techniques for a specific bottleneck.
The paper tackles the problem of estimating long-term treatment effects in scenarios where treatments have lasting, evolving impacts, which is difficult with traditional randomized experiments. It introduces a reinforcement learning approach for nonstationary Markov processes and demonstrates promising results on synthetic and online store datasets.
Randomized experiments (a.k.a. A/B tests) are a powerful tool for estimating treatment effects, to inform decisions making in business, healthcare and other applications. In many problems, the treatment has a lasting effect that evolves over time. A limitation with randomized experiments is that they do not easily extend to measure long-term effects, since running long experiments is time-consuming and expensive. In this paper, we take a reinforcement learning (RL) approach that estimates the average reward in a Markov process. Motivated by real-world scenarios where the observed state transition is nonstationary, we develop a new algorithm for a class of nonstationary problems, and demonstrate promising results in two synthetic datasets and one online store dataset.