Online learning with Corrupted context: Corrupted Contextual Bandits
This addresses a specific issue in online decision-making applications like clinical trials and ad recommendations, but it is incremental as it builds on existing bandit methods.
The paper tackles the problem of corrupted context in online learning by proposing a hybrid method that combines contextual and multi-armed bandits, enabling learning from all iterations, including those with useless context, and shows promising empirical results on real-life datasets.
We consider a novel variant of the contextual bandit problem (i.e., the multi-armed bandit with side-information, or context, available to a decision-maker) where the context used at each decision may be corrupted ("useless context"). This new problem is motivated by certain on-line settings including clinical trial and ad recommendation applications. In order to address the corrupted-context setting,we propose to combine the standard contextual bandit approach with a classical multi-armed bandit mechanism. Unlike standard contextual bandit methods, we are able to learn from all iteration, even those with corrupted context, by improving the computing of the expectation for each arm. Promising empirical results are obtained on several real-life datasets.