FedSlate:A Federated Deep Reinforcement Learning Recommender System
This work addresses privacy and cross-platform learning issues in recommendation systems for users and platforms, though it is incremental as it combines existing federated learning and reinforcement learning techniques.
The paper tackles the challenge of optimizing long-term user engagement in recommendation systems across multiple platforms while addressing privacy and communication costs by proposing FedSlate, a federated reinforcement learning algorithm. Experimental results show FedSlate outperforms state-of-the-art baselines in various settings and enables learning in scenarios where other methods fail.
Reinforcement learning methods have been used to optimize long-term user engagement in recommendation systems. However, existing reinforcement learning-based recommendation systems do not fully exploit the relevance of individual user behavior across different platforms. One potential solution is to aggregate data from various platforms in a centralized location and use the aggregated data for training. However, this approach raises economic and legal concerns, including increased communication costs and potential threats to user privacy. To address these challenges, we propose \textbf{FedSlate}, a federated reinforcement learning recommendation algorithm that effectively utilizes information that is prohibited from being shared at a legal level. We employ the SlateQ algorithm to assist FedSlate in learning users' long-term behavior and evaluating the value of recommended content. We extend the existing application scope of recommendation systems from single-user single-platform to single-user multi-platform and address cross-platform learning challenges by introducing federated learning. We use RecSim to construct a simulation environment for evaluating FedSlate and compare its performance with state-of-the-art benchmark recommendation models. Experimental results demonstrate the superior effects of FedSlate over baseline methods in various environmental settings, and FedSlate facilitates the learning of recommendation strategies in scenarios where baseline methods are completely inapplicable. Code is available at \textit{https://github.com/TianYaDY/FedSlate}.