Learning List-wise Representation in Reinforcement Learning for Ads Allocation with Multiple Auxiliary Tasks
This work addresses ads allocation in recommendation platforms like e-commerce, offering incremental improvements by enhancing list-wise representation learning with auxiliary tasks.
The paper tackles the challenge of learning list-wise representations in reinforcement learning for ads allocation, which suffers from high-dimensional state-action spaces and poor sample efficiency, and proposes using multiple auxiliary tasks to improve representation learning, resulting in higher revenue on the Meituan platform compared to state-of-the-art baselines.
With the recent prevalence of reinforcement learning (RL), there have been tremendous interests in utilizing RL for ads allocation in recommendation platforms (e.g., e-commerce and news feed sites). To achieve better allocation, the input of recent RL-based ads allocation methods is upgraded from point-wise single item to list-wise item arrangement. However, this also results in a high-dimensional space of state-action pairs, making it difficult to learn list-wise representations with good generalization ability. This further hinders the exploration of RL agents and causes poor sample efficiency. To address this problem, we propose a novel RL-based approach for ads allocation which learns better list-wise representations by leveraging task-specific signals on Meituan food delivery platform. Specifically, we propose three different auxiliary tasks based on reconstruction, prediction, and contrastive learning respectively according to prior domain knowledge on ads allocation. We conduct extensive experiments on Meituan food delivery platform to evaluate the effectiveness of the proposed auxiliary tasks. Both offline and online experimental results show that the proposed method can learn better list-wise representations and achieve higher revenue for the platform compared to the state-of-the-art baselines.