On the Relationship Between Active Inference and Control as Inference
This work clarifies the relationship between two emerging frameworks in brain sciences and reinforcement learning, which is incremental as it builds on existing theories without introducing new methods.
The paper formally compares Active Inference (AIF) and Control-as-Inference (CAI), showing that their key difference lies in how value is integrated into generative models, and suggests ways these frameworks can mutually inform each other.
Active Inference (AIF) is an emerging framework in the brain sciences which suggests that biological agents act to minimise a variational bound on model evidence. Control-as-Inference (CAI) is a framework within reinforcement learning which casts decision making as a variational inference problem. While these frameworks both consider action selection through the lens of variational inference, their relationship remains unclear. Here, we provide a formal comparison between them and demonstrate that the primary difference arises from how value is incorporated into their respective generative models. In the context of this comparison, we highlight several ways in which these frameworks can inform one another.