Latent Bandits Revisited
This work addresses the challenge of balancing online and offline learning in practical applications like recommender systems, though it is incremental as it builds on existing bandit frameworks.
The authors tackled the latent bandit problem, where an agent must identify an unknown latent state to optimize actions, by proposing contextual algorithms based on UCB and Thompson sampling that account for model uncertainty and misspecification, resulting in lower regret than classic policies when latent states are fewer than actions.
A latent bandit problem is one in which the learning agent knows the arm reward distributions conditioned on an unknown discrete latent state. The primary goal of the agent is to identify the latent state, after which it can act optimally. This setting is a natural midpoint between online and offline learning---complex models can be learned offline with the agent identifying latent state online---of practical relevance in, say, recommender systems. In this work, we propose general algorithms for this setting, based on both upper confidence bounds (UCBs) and Thompson sampling. Our methods are contextual and aware of model uncertainty and misspecification. We provide a unified theoretical analysis of our algorithms, which have lower regret than classic bandit policies when the number of latent states is smaller than actions. A comprehensive empirical study showcases the advantages of our approach.