Bandit problems with fidelity rewards
This work addresses a novel variant of the multi-armed bandit problem, which could impact reinforcement learning and decision-making systems, though it appears incremental as it builds on existing bandit frameworks.
The paper tackles the fidelity bandits problem, where rewards are augmented by loyalty-based bonuses, and introduces two fidelity models (loyalty-points and subscription) for both stochastic and adversarial settings. It provides algorithms with sublinear regret bounds for some models and worst-case lower bounds for others.
The fidelity bandits problem is a variant of the $K$-armed bandit problem in which the reward of each arm is augmented by a fidelity reward that provides the player with an additional payoff depending on how 'loyal' the player has been to that arm in the past. We propose two models for fidelity. In the loyalty-points model the amount of extra reward depends on the number of times the arm has previously been played. In the subscription model the additional reward depends on the current number of consecutive draws of the arm. We consider both stochastic and adversarial problems. Since single-arm strategies are not always optimal in stochastic problems, the notion of regret in the adversarial setting needs careful adjustment. We introduce three possible notions of regret and investigate which can be bounded sublinearly. We study in detail the special cases of increasing, decreasing and coupon (where the player gets an additional reward after every $m$ plays of an arm) fidelity rewards. For the models which do not necessarily enjoy sublinear regret, we provide a worst case lower bound. For those models which exhibit sublinear regret, we provide algorithms and bound their regret.