Bounded Rationality in Las Vegas: Probabilistic Finite Automata PlayMulti-Armed Bandits
This work addresses the problem of understanding human decision-making under computational limitations for researchers in economics and AI, though it is incremental in applying PFAs to bandits.
The paper tackles the problem of modeling bounded rationality in multi-armed bandits by proposing a simple strategy implementable by a probabilistic finite automaton (PFA), showing that with many states it performs near-optimally and degrades gracefully with fewer states, while exhibiting human-like biases such as optimism and negativity.
While traditional economics assumes that humans are fully rational agents who always maximize their expected utility, in practice, we constantly observe apparently irrational behavior. One explanation is that people have limited computational power, so that they are, quite rationally, making the best decisions they can, given their computational limitations. To test this hypothesis, we consider the multi-armed bandit (MAB) problem. We examine a simple strategy for playing an MAB that can be implemented easily by a probabilistic finite automaton (PFA). Roughly speaking, the PFA sets certain expectations, and plays an arm as long as it meets them. If the PFA has sufficiently many states, it performs near-optimally. Its performance degrades gracefully as the number of states decreases. Moreover, the PFA acts in a "human-like" way, exhibiting a number of standard human biases, like an optimism bias and a negativity bias.