Whence the Expected Free Energy?
This work addresses a foundational mathematical ambiguity in active inference theory, which is crucial for researchers developing AI agents that balance exploration and exploitation.
The paper clarifies that the Expected Free Energy (EFE) in active inference is not simply a future extension of Variational Free Energy, and introduces a new objective called Free-Energy of the Expected Future (FEEF) that combines epistemic value with a clear mathematical basis as a divergence measure.
The Expected Free Energy (EFE) is a central quantity in the theory of active inference. It is the quantity that all active inference agents are mandated to minimize through action, and its decomposition into extrinsic and intrinsic value terms is key to the balance of exploration and exploitation that active inference agents evince. Despite its importance, the mathematical origins of this quantity and its relation to the Variational Free Energy (VFE) remain unclear. In this paper, we investigate the origins of the EFE in detail and show that it is not simply "the free energy in the future". We present a functional that we argue is the natural extension of the VFE, but which actively discourages exploratory behaviour, thus demonstrating that exploration does not directly follow from free energy minimization into the future. We then develop a novel objective, the Free-Energy of the Expected Future (FEEF), which possesses both the epistemic component of the EFE as well as an intuitive mathematical grounding as the divergence between predicted and desired futures.