Adaptive Instrument Design for Indirect Experiments
This work addresses the challenge of estimating treatment effects when randomized control trials are not feasible, offering a practical solution for researchers and practitioners in fields like healthcare or social sciences, though it is incremental as it extends adaptive design concepts to indirect experiments.
The paper tackles the problem of low sample efficiency in indirect experiments by adaptively designing data collection policies over instrumental variables, resulting in a method that significantly improves efficiency across various real-world inspired domains.
Indirect experiments provide a valuable framework for estimating treatment effects in situations where conducting randomized control trials (RCTs) is impractical or unethical. Unlike RCTs, indirect experiments estimate treatment effects by leveraging (conditional) instrumental variables, enabling estimation through encouragement and recommendation rather than strict treatment assignment. However, the sample efficiency of such estimators depends not only on the inherent variability in outcomes but also on the varying compliance levels of users with the instrumental variables and the choice of estimator being used, especially when dealing with numerous instrumental variables. While adaptive experiment design has a rich literature for direct experiments, in this paper we take the initial steps towards enhancing sample efficiency for indirect experiments by adaptively designing a data collection policy over instrumental variables. Our main contribution is a practical computational procedure that utilizes influence functions to search for an optimal data collection policy, minimizing the mean-squared error of the desired (non-linear) estimator. Through experiments conducted in various domains inspired by real-world applications, we showcase how our method can significantly improve the sample efficiency of indirect experiments.