Information Theoretically Aided Reinforcement Learning for Embodied Agents
This work addresses the problem of local optima in reinforcement learning for embodied agents, offering an incremental improvement for robotics and AI applications.
The paper tackled the challenge of rugged reward functions in reinforcement learning for embodied agents by incorporating an intrinsic reward based on mutual information of sensor readings, which smoothed the optimization landscape and significantly improved policy gradient optimization for locomotion in complex morphologies.
Reinforcement learning for embodied agents is a challenging problem. The accumulated reward to be optimized is often a very rugged function, and gradient methods are impaired by many local optimizers. We demonstrate, in an experimental setting, that incorporating an intrinsic reward can smoothen the optimization landscape while preserving the global optimizers of interest. We show that policy gradient optimization for locomotion in a complex morphology is significantly improved when supplementing the extrinsic reward by an intrinsic reward defined in terms of the mutual information of time consecutive sensor readings.