A Simple Model to Estimate Sharing Effects in Social Networks
This addresses the challenge of causal inference for technology companies conducting A/B tests in social networks, though it is incremental as it builds on existing methods with a new model.
The paper tackled the problem of estimating treatment effects in social networks where user interactions cause interference, by proposing a simple MDP-based model for sharing behavior and deriving an unbiased estimator. The result showed that this estimator outperformed existing methods by a significant margin in synthetic experiments.
Randomised Controlled Trials (RCTs) are the gold standard for estimating treatment effects across many fields of science. Technology companies have adopted A/B-testing methods as a modern RCT counterpart, where end-users are randomly assigned various system variants and user behaviour is tracked continuously. The objective is then to estimate the causal effect that the treatment variant would have on certain metrics of interest to the business. When the outcomes for randomisation units -- end-users in this case -- are not statistically independent, this obfuscates identifiability of treatment effects, and harms decision-makers' observability of the system. Social networks exemplify this, as they are designed to promote inter-user interactions. This interference by design notoriously complicates measurement of, e.g., the effects of sharing. In this work, we propose a simple Markov Decision Process (MDP)-based model describing user sharing behaviour in social networks. We derive an unbiased estimator for treatment effects under this model, and demonstrate through reproducible synthetic experiments that it outperforms existing methods by a significant margin.