Conditionally Risk-Averse Contextual Bandits
This addresses the need for risk-averse decision-making in applications where average-case guarantees are insufficient, such as in critical systems, though it appears incremental by extending contextual bandits to incorporate risk aversion.
The paper tackles the problem of designing a contextual bandit algorithm that avoids worst-case outcomes in risk-averse scenarios, achieving the first such algorithm with an online regret guarantee and demonstrating its effectiveness in applications like dynamic pricing and a production exascale data processing system.
Contextual bandits with average-case statistical guarantees are inadequate in risk-averse situations because they might trade off degraded worst-case behaviour for better average performance. Designing a risk-averse contextual bandit is challenging because exploration is necessary but risk-aversion is sensitive to the entire distribution of rewards; nonetheless we exhibit the first risk-averse contextual bandit algorithm with an online regret guarantee. We conduct experiments from diverse scenarios where worst-case outcomes should be avoided, from dynamic pricing, inventory management, and self-tuning software; including a production exascale data processing system.