Towards Domain Adaptive Neural Contextual Bandits
This addresses the challenge of costly feedback collection in decision-making problems like drug testing, though it is incremental as it builds on existing contextual bandit methods.
The paper tackles the problem of adapting contextual bandit algorithms across domains with distribution shifts, such as from mice to humans, by introducing a domain adaptation method that learns a bandit model for the target domain using source domain feedback, achieving sub-linear regret bounds and outperforming state-of-the-art algorithms on real-world datasets.
Contextual bandit algorithms are essential for solving real-world decision making problems. In practice, collecting a contextual bandit's feedback from different domains may involve different costs. For example, measuring drug reaction from mice (as a source domain) and humans (as a target domain). Unfortunately, adapting a contextual bandit algorithm from a source domain to a target domain with distribution shift still remains a major challenge and largely unexplored. In this paper, we introduce the first general domain adaptation method for contextual bandits. Our approach learns a bandit model for the target domain by collecting feedback from the source domain. Our theoretical analysis shows that our algorithm maintains a sub-linear regret bound even adapting across domains. Empirical results show that our approach outperforms the state-of-the-art contextual bandit algorithms on real-world datasets.