Toward Simulating Environments in Reinforcement Learning Based Recommendations
This addresses the challenge of reducing time, effort, and negative user impacts in developing RL-based recommender systems, though it is incremental as it builds on existing GAN methods for simulation.
The paper tackles the problem of training and evaluating reinforcement learning-based recommender systems by developing a user simulator using Generative Adversarial Networks to mimic real user behaviors, with experimental results on real-world e-commerce data demonstrating its effectiveness.
With the recent advances in Reinforcement Learning (RL), there have been tremendous interests in employing RL for recommender systems. However, directly training and evaluating a new RL-based recommendation algorithm needs to collect users' real-time feedback in the real system, which is time and efforts consuming and could negatively impact on users' experiences. Thus, it calls for a user simulator that can mimic real users' behaviors where we can pre-train and evaluate new recommendation algorithms. Simulating users' behaviors in a dynamic system faces immense challenges -- (i) the underlining item distribution is complex, and (ii) historical logs for each user are limited. In this paper, we develop a user simulator base on Generative Adversarial Network (GAN). To be specific, the generator captures the underlining distribution of users' historical logs and generates realistic logs that can be considered as augmentations of real logs; while the discriminator not only distinguishes real and fake logs but also predicts users' behaviors. The experimental results based on real-world e-commerce data demonstrate the effectiveness of the proposed simulator.