LinkLouvain: Link-Aware A/B Testing and Its Application on Online Marketing Campaign
This work is significant for online marketing campaign managers who need accurate A/B testing results despite user interaction interference.
This paper addresses the challenge of accurately measuring the average treatment effect (ATE) in online marketing campaigns using A/B testing, where user interactions can cause interference. They propose LinkLouvain, a network A/B testing method that minimizes graph interference to provide accurate and sound ATE estimates.
A lot of online marketing campaigns aim to promote user interaction. The average treatment effect (ATE) of campaign strategies need to be monitored throughout the campaign. A/B testing is usually conducted for such needs, whereas the existence of user interaction can introduce interference to normal A/B testing. With the help of link prediction, we design a network A/B testing method LinkLouvain to minimize graph interference and it gives an accurate and sound estimate of the campaign's ATE. In this paper, we analyze the network A/B testing problem under a real-world online marketing campaign, describe our proposed LinkLouvain method, and evaluate it on real-world data. Our method achieves significant performance compared with others and is deployed in the online marketing campaign.