Hybrid Transfer in Deep Reinforcement Learning for Ads Allocation
This work addresses data scarcity in ads allocation for e-commerce platforms, but it is incremental as it builds on existing transfer learning methods.
The paper tackles the challenge of low data coverage in ads allocation for e-commerce platforms by proposing a hybrid transfer method that transfers samples and knowledge from data-rich to data-poor entrances, resulting in increased platform revenue as demonstrated in offline and online experiments.
Ads allocation, which involves allocating ads and organic items to limited slots in feed with the purpose of maximizing platform revenue, has become a research hotspot. Notice that, e-commerce platforms usually have multiple entrances for different categories and some entrances have few visits. Data from these entrances has low coverage, which makes it difficult for the agent to learn. To address this challenge, we propose Similarity-based Hybrid Transfer for Ads Allocation (SHTAA), which effectively transfers samples as well as knowledge from data-rich entrance to data-poor entrance. Specifically, we define an uncertainty-aware similarity for MDP to estimate the similarity of MDP for different entrances. Based on this similarity, we design a hybrid transfer method, including instance transfer and strategy transfer, to efficiently transfer samples and knowledge from one entrance to another. Both offline and online experiments on Meituan food delivery platform demonstrate that the proposed method could achieve better performance for data-poor entrance and increase the revenue for the platform.