Frugal inference for control
This work provides a foundation for resource-efficient control under uncertainty, potentially impacting both AI systems and neuroscience, though it is incremental in extending existing POMDP frameworks.
The paper tackles the problem of balancing utility maximization with resource efficiency in partially observable environments by developing a POMDP framework that treats inference as an optimizable resource, revealing a phase transition where frugal inference strategies leave uncertainty unresolved to facilitate adaptation.
A key challenge in advancing artificial intelligence is achieving the right balance between utility maximization and resource use by both external movement and internal computation. While this trade-off has been studied in fully observable settings, our understanding of resource efficiency in partially observable environments remains limited. Motivated by this challenge, we develop a version of the POMDP framework where the information gained through inference is treated as a resource that must be optimized alongside task performance and motion effort. By solving this problem in environments described by linear-Gaussian dynamics, we uncover fundamental principles of resource efficiency. Our study reveals a phase transition in the inference, switching from a Bayes-optimal approach to one that strategically leaves some uncertainty unresolved. This frugal behavior gives rise to a structured family of equally effective strategies, facilitating adaptation to later objectives and constraints overlooked during the original optimization. We illustrate the applicability of our framework and the generality of the principles we derived using two nonlinear tasks. Overall, this work provides a foundation for a new type of rational computation that both brains and machines could use for effective but resource-efficient control under uncertainty.