Jointly Learning to Recommend and Advertise
This work addresses the problem of improving overall platform performance for online recommendation platforms like e-commerce and news feeds, though it is incremental as it builds on existing reinforcement learning techniques.
The paper tackles the suboptimal performance from separately optimizing recommendation and advertising strategies by proposing a two-level reinforcement learning framework that jointly optimizes both, balancing immediate ad revenue with long-term user experience, and demonstrates its effectiveness on real-world data.
Online recommendation and advertising are two major income channels for online recommendation platforms (e.g. e-commerce and news feed site). However, most platforms optimize recommending and advertising strategies by different teams separately via different techniques, which may lead to suboptimal overall performances. To this end, in this paper, we propose a novel two-level reinforcement learning framework to jointly optimize the recommending and advertising strategies, where the first level generates a list of recommendations to optimize user experience in the long run; then the second level inserts ads into the recommendation list that can balance the immediate advertising revenue from advertisers and the negative influence of ads on long-term user experience. To be specific, the first level tackles high combinatorial action space problem that selects a subset items from the large item space; while the second level determines three internally related tasks, i.e., (i) whether to insert an ad, and if yes, (ii) the optimal ad and (iii) the optimal location to insert. The experimental results based on real-world data demonstrate the effectiveness of the proposed framework. We have released the implementation code to ease reproductivity.