A Simple Reduction Scheme for Constrained Contextual Bandits with Adversarial Contexts via Regression
This work addresses constrained decision-making under adversarial conditions for machine learning applications, offering a modular approach that builds incrementally on existing frameworks.
The authors tackled the problem of constrained contextual bandits with adversarial contexts by proposing a reduction scheme using online regression oracles, achieving improved guarantees for regret and constraint violation compared to prior stochastic context methods.
We study constrained contextual bandits (CCB) with adversarially chosen contexts, where each action yields a random reward and incurs a random cost. We adopt the standard realizability assumption: conditioned on the observed context, rewards and costs are drawn independently from fixed distributions whose expectations belong to known function classes. We consider the continuing setting, in which the algorithm operates over the entire horizon even after the budget is exhausted. In this setting, the objective is to simultaneously control regret and cumulative constraint violation. Building on the seminal SquareCB framework of Foster et al. (2018), we propose a simple and modular algorithmic scheme that leverages online regression oracles to reduce the constrained problem to a standard unconstrained contextual bandit problem with adaptively defined surrogate reward functions. In contrast to most prior work on CCB, which focuses on stochastic contexts, our reduction yields improved guarantees for the more general adversarial context setting, together with a compact and transparent analysis.