Active Inference for Stochastic Control
This work addresses the computational limitations of active inference for stochastic control, offering a solution for researchers and practitioners in robotics or AI needing robust control under uncertainty, though it is incremental as it builds on recent planning algorithm advancements.
The paper tackled the challenge of applying active inference to stochastic control problems by simulating a complex windy grid-world task with environment stochasticity, learning of transition dynamics, and partial observability, and demonstrated that active inference outperforms reinforcement learning in both deterministic and stochastic settings.
Active inference has emerged as an alternative approach to control problems given its intuitive (probabilistic) formalism. However, despite its theoretical utility, computational implementations have largely been restricted to low-dimensional, deterministic settings. This paper highlights that this is a consequence of the inability to adequately model stochastic transition dynamics, particularly when an extensive policy (i.e., action trajectory) space must be evaluated during planning. Fortunately, recent advancements propose a modified planning algorithm for finite temporal horizons. We build upon this work to assess the utility of active inference for a stochastic control setting. For this, we simulate the classic windy grid-world task with additional complexities, namely: 1) environment stochasticity; 2) learning of transition dynamics; and 3) partial observability. Our results demonstrate the advantage of using active inference, compared to reinforcement learning, in both deterministic and stochastic settings.