Evolution of Information in Interactive Decision Making: A Case Study for Multi-Armed Bandits
This provides theoretical insights into the interplay between information and learning in interactive systems, but it is incremental as it focuses on a specific case study.
The paper tackled the evolution of information in interactive decision-making by analyzing a stochastic multi-armed bandit problem with a unique optimal arm, revealing distinct growth phases in mutual information and showing that optimal learning does not require maximizing information gain.
We study the evolution of information in interactive decision making through the lens of a stochastic multi-armed bandit problem. Focusing on a fundamental example where a unique optimal arm outperforms the rest by a fixed margin, we characterize the optimal success probability and mutual information over time. Our findings reveal distinct growth phases in mutual information -- initially linear, transitioning to quadratic, and finally returning to linear -- highlighting curious behavioral differences between interactive and non-interactive environments. In particular, we show that optimal success probability and mutual information can be decoupled, where achieving optimal learning does not necessarily require maximizing information gain. These findings shed new light on the intricate interplay between information and learning in interactive decision making.