Leveraging User-Triggered Supervision in Contextual Bandits
This work addresses a specific challenge in interactive systems like text prediction, offering incremental improvements for better user experience.
The paper tackles the problem of improving contextual bandit algorithms by incorporating user-triggered supervision, where users provide feedback only in certain contexts, and shows improved regret guarantees under various conditions.
We study contextual bandit (CB) problems, where the user can sometimes respond with the best action in a given context. Such an interaction arises, for example, in text prediction or autocompletion settings, where a poor suggestion is simply ignored and the user enters the desired text instead. Crucially, this extra feedback is user-triggered on only a subset of the contexts. We develop a new framework to leverage such signals, while being robust to their biased nature. We also augment standard CB algorithms to leverage the signal, and show improved regret guarantees for the resulting algorithms under a variety of conditions on the helpfulness of and bias inherent in this feedback.