Minimizing Interference and Selection Bias in Network Experiment Design
This addresses the problem of biased causal effect estimation in network experiments for researchers and practitioners, representing an incremental improvement over prior methods.
The paper tackles the problem of selection bias in network A/B testing caused by cluster randomization, proposing a framework that jointly minimizes interference and selection bias. Experiments on real-world datasets show it leads to significantly lower error in causal effect estimation compared to existing solutions.
Current approaches to A/B testing in networks focus on limiting interference, the concern that treatment effects can "spill over" from treatment nodes to control nodes and lead to biased causal effect estimation. Prominent methods for network experiment design rely on two-stage randomization, in which sparsely-connected clusters are identified and cluster randomization dictates the node assignment to treatment and control. Here, we show that cluster randomization does not ensure sufficient node randomization and it can lead to selection bias in which treatment and control nodes represent different populations of users. To address this problem, we propose a principled framework for network experiment design which jointly minimizes interference and selection bias. We introduce the concepts of edge spillover probability and cluster matching and demonstrate their importance for designing network A/B testing. Our experiments on a number of real-world datasets show that our proposed framework leads to significantly lower error in causal effect estimation than existing solutions.