A Large Scale Heterogeneous Treatment Effect Estimation Framework and Its Applications of Users' Journey at Snap
This work addresses the challenge of personalizing treatment effects for users in large-scale applications like advertising at Snap, though it appears incremental as it builds on existing HTE methods with industrial optimizations.
The paper tackles the problem of estimating heterogeneous treatment effects (HTE) for Snapchat users by developing a large-scale industrial framework that processes data from hundreds of millions of users, resulting in stable estimates and uncovering latent user characteristics. An online A/B test using influenceability scores for ad targeting showed an improvement in key business metrics more than six times larger than typical significance levels.
Heterogeneous Treatment Effect (HTE) and Conditional Average Treatment Effect (CATE) models relax the assumption that treatment effects are the same for every user. We present a large scale industrial framework for estimating HTE using experimental data from hundreds of millions of Snapchat users. By combining results across many experiments, the framework uncovers latent user characteristics that were previously unmeasurable and produces stable treatment effect estimates at scale. We describe the core components that enabled this system, including experiment selection, base learner design, and incremental training. We also highlight two applications: user influenceability to ads and user sensitivity to ads. An online A/B test using influenceability scores for targeting showed an improvement on key business metrics that is more than six times larger than what is typically considered significant.