Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback
This work addresses the challenge of robustly combining different data sources for contextual bandit learning, which is incremental as it builds on existing bandit methods.
The paper tackled the problem of learning from both supervised and contextual bandit data with potentially misaligned cost distributions, and it developed no-regret algorithms that proved feasible and helpful in practice across multiple datasets.
We investigate the feasibility of learning from a mix of both fully-labeled supervised data and contextual bandit data. We specifically consider settings in which the underlying learning signal may be different between these two data sources. Theoretically, we state and prove no-regret algorithms for learning that is robust to misaligned cost distributions between the two sources. Empirically, we evaluate some of these algorithms on a large selection of datasets, showing that our approach is both feasible and helpful in practice.