A Feedback Shift Correction in Predicting Conversion Rates under Delayed Feedback
This work addresses a specific issue in online advertising for advertisers and platforms, offering an incremental improvement over prior methods.
The paper tackles the problem of delayed feedback in conversion rate prediction for display advertising, where some positive instances are mislabeled as negative during training, leading to a feedback shift. The authors propose an importance weight approach to correct this shift and demonstrate its effectiveness through offline and online experiments, showing it outperforms existing methods.
In display advertising, predicting the conversion rate, that is, the probability that a user takes a predefined action on an advertiser's website, such as purchasing goods is fundamental in estimating the value of displaying the advertisement. However, there is a relatively long time delay between a click and its resultant conversion. Because of the delayed feedback, some positive instances at the training period are labeled as negative because some conversions have not yet occurred when training data are gathered. As a result, the conditional label distributions differ between the training data and the production environment. This situation is referred to as a feedback shift. We address this problem by using an importance weight approach typically used for covariate shift correction. We prove its consistency for the feedback shift. Results in both offline and online experiments show that our proposed method outperforms the existing method.