Conformal Bandits: Bringing statistical validity and reward efficiency to the small-gap regime
This addresses the need for statistically valid decision-making in sequential problems like finance where small gaps make traditional bandit methods ineffective.
The paper tackles the problem of bandit algorithms lacking statistical guarantees in small-gap regimes where reward differences are minimal, by introducing Conformal Bandits which integrates Conformal Prediction to provide finite-time coverage guarantees while maintaining regret efficiency, demonstrating in portfolio allocation applications that it achieves nominal coverage where classical methods fail and improves risk-adjusted returns.
We introduce Conformal Bandits, a novel framework integrating Conformal Prediction (CP) into bandit problems, a classic paradigm for sequential decision-making under uncertainty. Traditional regret-minimisation bandit strategies like Thompson Sampling and Upper Confidence Bound (UCB) typically rely on distributional assumptions or asymptotic guarantees; further, they remain largely focused on regret, neglecting their statistical properties. We address this gap. Through the adoption of CP, we bridge the regret-minimising potential of a decision-making bandit policy with statistical guarantees in the form of finite-time prediction coverage. We demonstrate the potential of it Conformal Bandits through simulation studies and an application to portfolio allocation, a typical small-gap regime, where differences in arm rewards are far too small for classical policies to achieve optimal regret bounds in finite sample. Motivated by this, we showcase our framework's practical advantage in terms of regret in small-gap settings, as well as its added value in achieving nominal coverage guarantees where classical UCB policies fail. Focusing on our application of interest, we further illustrate how integrating hidden Markov models to capture the regime-switching behaviour of financial markets, enhances the exploration-exploitation trade-off, and translates into higher risk-adjusted regret efficiency returns, while preserving coverage guarantees.