Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation
This work addresses the challenge of reducing interaction costs in online recommendation systems, though it appears incremental as it builds on existing model-based and adversarial training methods.
The paper tackles the problem of expensive model learning in model-free reinforcement learning for recommender systems by proposing a model-based approach using a generative adversarial network to model user-agent interactions for offline policy learning, and demonstrates its effectiveness through theoretical analysis and empirical evaluations.
Reinforcement learning is well suited for optimizing policies of recommender systems. Current solutions mostly focus on model-free approaches, which require frequent interactions with the real environment, and thus are expensive in model learning. Offline evaluation methods, such as importance sampling, can alleviate such limitations, but usually request a large amount of logged data and do not work well when the action space is large. In this work, we propose a model-based reinforcement learning solution which models user-agent interaction for offline policy learning via a generative adversarial network. To reduce bias in the learned model and policy, we use a discriminator to evaluate the quality of generated data and scale the generated rewards. Our theoretical analysis and empirical evaluations demonstrate the effectiveness of our solution in learning policies from the offline and generated data.