Cross DQN: Cross Deep Q Network for Ads Allocation in Feed
This work addresses a domain-specific challenge in e-commerce feed optimization, offering an incremental improvement over existing methods for ads allocation.
The paper tackles the problem of allocating ads and organic items in e-commerce feeds to maximize revenue and user experience by modeling arrangement signals and controlling the percentage of ads exposed (PAE). The proposed Cross DQN model achieved higher revenue and better user experience than state-of-the-art baselines in offline experiments and was successfully deployed on Meituan feed, serving over 300 million customers.
E-commerce platforms usually display a mixed list of ads and organic items in feed. One key problem is to allocate the limited slots in the feed to maximize the overall revenue as well as improve user experience, which requires a good model for user preference. Instead of modeling the influence of individual items on user behaviors, the arrangement signal models the influence of the arrangement of items and may lead to a better allocation strategy. However, most of previous strategies fail to model such a signal and therefore result in suboptimal performance. In addition, the percentage of ads exposed (PAE) is an important indicator in ads allocation. Excessive PAE hurts user experience while too low PAE reduces platform revenue. Therefore, how to constrain the PAE within a certain range while keeping personalized recommendation under the PAE constraint is a challenge. In this paper, we propose Cross Deep Q Network (Cross DQN) to extract the crucial arrangement signal by crossing the embeddings of different items and modeling the crossed sequence by multi-channel attention. Besides, we propose an auxiliary loss for batch-level constraint on PAE to tackle the above-mentioned challenge. Our model results in higher revenue and better user experience than state-of-the-art baselines in offline experiments. Moreover, our model demonstrates a significant improvement in the online A/B test and has been fully deployed on Meituan feed to serve more than 300 millions of customers.