An Adversarial Interpretation of Information-Theoretic Bounded Rationality
This provides a theoretical bridge between information-theoretic bounded rationality and game theory, which is incremental but clarifies foundational relationships in decision-making models.
The paper tackles the problem of connecting free energy optimization in bounded rationality to adversarial environments by showing that it is equivalent to a game between an agent and an imaginary adversary, where the adversary's optimal strategy induces indifference among choices, linking it to Nash equilibrium.
Recently, there has been a growing interest in modeling planning with information constraints. Accordingly, an agent maximizes a regularized expected utility known as the free energy, where the regularizer is given by the information divergence from a prior to a posterior policy. While this approach can be justified in various ways, including from statistical mechanics and information theory, it is still unclear how it relates to decision-making against adversarial environments. This connection has previously been suggested in work relating the free energy to risk-sensitive control and to extensive form games. Here, we show that a single-agent free energy optimization is equivalent to a game between the agent and an imaginary adversary. The adversary can, by paying an exponential penalty, generate costs that diminish the decision maker's payoffs. It turns out that the optimal strategy of the adversary consists in choosing costs so as to render the decision maker indifferent among its choices, which is a definining property of a Nash equilibrium, thus tightening the connection between free energy optimization and game theory.