BanditQ: Fair Bandits with Guaranteed Rewards
This addresses fairness in bandit algorithms for applications requiring equitable resource allocation, though it is incremental as it builds on existing adversarial bandit and queueing techniques.
The paper tackles the unfairness of classic multi-armed bandit algorithms by introducing a fair prediction problem with guaranteed minimum reward rates for each arm, proposing BanditQ, which achieves target rates with a regret and violation penalty of at most O(T^{3/4}) in bandit feedback and O(√T) in full-information settings.
Classic no-regret multi-armed bandit algorithms, including the Upper Confidence Bound (UCB), Hedge, and EXP3, are inherently unfair by design. Their unfairness stems from their objective of playing the most rewarding arm as frequently as possible while ignoring the rest. In this paper, we consider a fair prediction problem in the stochastic setting with a guaranteed minimum rate of accrual of rewards for each arm. We study the problem in both full-information and bandit feedback settings. Combining queueing-theoretic techniques with adversarial bandits, we propose a new online policy, called BanditQ, that achieves the target reward rates while conceding a regret and target rate violation penalty of at most $O(T^{\frac{3}{4}}).$ The regret bound in the full-information setting can be further improved to $O(\sqrt{T})$ under either a monotonicity assumption or when considering time-averaged regret. The proposed policy is efficient and admits a black-box reduction from the fair prediction problem to the standard adversarial MAB problem. The analysis of the BanditQ policy involves a new self-bounding inequality, which might be of independent interest.