Context Uncertainty in Contextual Bandits with Applications to Recommender Systems
This addresses the issue of limited exploration in recommender systems for users, though it is incremental in nature.
The paper tackles the problem of balancing relevance and exploration in recommender systems by proposing recurrent exploration networks (REN), which achieve satisfactory long-term rewards and outperform state-of-the-art models on synthetic and real-world datasets.
Recurrent neural networks have proven effective in modeling sequential user feedbacks for recommender systems. However, they usually focus solely on item relevance and fail to effectively explore diverse items for users, therefore harming the system performance in the long run. To address this problem, we propose a new type of recurrent neural networks, dubbed recurrent exploration networks (REN), to jointly perform representation learning and effective exploration in the latent space. REN tries to balance relevance and exploration while taking into account the uncertainty in the representations. Our theoretical analysis shows that REN can preserve the rate-optimal sublinear regret even when there exists uncertainty in the learned representations. Our empirical study demonstrates that REN can achieve satisfactory long-term rewards on both synthetic and real-world recommendation datasets, outperforming state-of-the-art models.