Vector preference-based contextual bandits under distributional shifts
This work addresses distribution shifts in multi-objective bandit problems, which is incremental as it extends existing vector reward settings to handle shifts.
The paper tackles the problem of contextual bandit learning under distribution shift with vector rewards ordered by a preference cone, proposing a policy that self-tunes to shifts and introducing preference-based regret to measure performance via Pareto front distances. It establishes upper bounds on regret that generalize no-shift cases and scale well with parameters.
We consider contextual bandit learning under distribution shift when reward vectors are ordered according to a given preference cone. We propose an adaptive-discretization and optimistic elimination based policy that self-tunes to the underlying distribution shift. To measure the performance of this policy, we introduce the notion of preference-based regret which measures the performance of a policy in terms of distance between Pareto fronts. We study the performance of this policy by establishing upper bounds on its regret under various assumptions on the nature of distribution shift. Our regret bounds generalize known results for the existing case of no distribution shift and vectorial reward settings, and scale gracefully with problem parameters in presence of distribution shifts.