Towards Fundamental Limits of Multi-armed Bandits with Random Walk Feedback
This addresses a novel feedback mechanism in bandit problems for researchers, but it is incremental as it extends existing frameworks.
The paper tackles a new Multi-Armed Bandit problem where arms are nodes in an unknown graph, with rewards based on random walk lengths, and shows it is not easier than standard MAB in information theory.
In this paper, we consider a new Multi-Armed Bandit (MAB) problem where arms are nodes in an unknown and possibly changing graph, and the agent (i) initiates random walks over the graph by pulling arms, (ii) observes the random walk trajectories, and (iii) receives rewards equal to the lengths of the walks. We provide a comprehensive understanding of this problem by studying both the stochastic and the adversarial setting. We show that this problem is not easier than a standard MAB in an information theoretical sense, although additional information is available through random walk trajectories. Behaviors of bandit algorithms on this problem are also studied.