Backprop-Free Reinforcement Learning with Active Neural Generative Coding
This work addresses the challenge of backpropagation-free learning in reinforcement learning, which could benefit applications requiring efficient and biologically plausible AI systems, though it appears incremental as it builds on existing cognitive theories and shows results on simple tasks.
The researchers tackled the problem of reinforcement learning without backpropagation by proposing active neural generative coding, a framework for learning action-driven generative models in dynamic environments, and demonstrated competitive performance with deep Q-learning on simple control problems.
In humans, perceptual awareness facilitates the fast recognition and extraction of information from sensory input. This awareness largely depends on how the human agent interacts with the environment. In this work, we propose active neural generative coding, a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. Specifically, we develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. We demonstrate on several simple control problems that our framework performs competitively with deep Q-learning. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.