Intrinsic Motivation in Dynamical Control Systems
This work addresses the challenge of designing practical artificial controllers with intrinsic motivation and linking animal behaviors to dynamical properties, representing a novel method for a known bottleneck.
The study tackled the problem of understanding intrinsic motivation in biological systems by proposing an information-theoretic approach based on maximizing empowerment, and it demonstrated success on benchmark control problems with a computationally efficient algorithm.
Biological systems often choose actions without an explicit reward signal, a phenomenon known as intrinsic motivation. The computational principles underlying this behavior remain poorly understood. In this study, we investigate an information-theoretic approach to intrinsic motivation, based on maximizing an agent's empowerment (the mutual information between its past actions and future states). We show that this approach generalizes previous attempts to formalize intrinsic motivation, and we provide a computationally efficient algorithm for computing the necessary quantities. We test our approach on several benchmark control problems, and we explain its success in guiding intrinsically motivated behaviors by relating our information-theoretic control function to fundamental properties of the dynamical system representing the combined agent-environment system. This opens the door for designing practical artificial, intrinsically motivated controllers and for linking animal behaviors to their dynamical properties.