Identifiable Latent Bandits: Leveraging observational data for personalized decision-making
This addresses the problem of sample-hungry sequential decision-making in domains like personalized medicine, where training from scratch per patient is infeasible, though it is incremental as it builds on latent bandits by adding identifiability.
The authors tackled the sample inefficiency of multi-armed bandits in personalized decision-making by proposing an identifiable latent bandit framework that learns from historical observational data, achieving optimal decisions with shorter exploration times than classical bandits.
Sequential decision-making algorithms such as multi-armed bandits can find optimal personalized decisions, but are notoriously sample-hungry. In personalized medicine, for example, training a bandit from scratch for every patient is typically infeasible, as the number of trials required is much larger than the number of decision points for a single patient. To combat this, latent bandits offer rapid exploration and personalization beyond what context variables alone can offer, provided that a latent variable model of problem instances can be learned consistently. However, existing works give no guidance as to how such a model can be found. In this work, we propose an identifiable latent bandit framework that leads to optimal decision-making with a shorter exploration time than classical bandits by learning from historical records of decisions and outcomes. Our method is based on nonlinear independent component analysis that provably identifies representations from observational data sufficient to infer optimal actions in new bandit instances. We verify this strategy in simulated and semi-synthetic environments, showing substantial improvement over online and offline learning baselines when identifying conditions are satisfied.