Distributional Active Inference
This work addresses sample inefficiency in reinforcement learning for robotics, offering a novel integration that could improve control in complex environments.
The paper tackled the dual challenge of efficient sensory information organization and far-sighted planning in robotic control by integrating active inference into distributional reinforcement learning, achieving performance advantages without requiring transition dynamics modeling.
Optimal control of complex environments with robotic systems faces two complementary and intertwined challenges: efficient organization of sensory state information and far-sighted action planning. Because the reinforcement learning framework addresses only the latter, it tends to deliver sample-inefficient solutions. Active inference is the state-of-the-art process theory that explains how biological brains handle this dual problem. However, its applications to artificial intelligence have thus far been limited to extensions of existing model-based approaches. We present a formal abstraction of reinforcement learning algorithms that spans model-based, distributional, and model-free approaches. This abstraction seamlessly integrates active inference into the distributional reinforcement learning framework, making its performance advantages accessible without transition dynamics modeling.