Mutual Information-based State-Control for Intrinsically Motivated Reinforcement Learning
This approach could benefit robotic systems by reducing reliance on external reward signals, though it appears incremental as it builds on existing intrinsic motivation methods.
The paper tackles the problem of enabling reinforcement learning agents to learn from internal drives rather than external rewards by proposing a mutual information-based intrinsic objective between goal states and controllable states, and demonstrates its efficacy in robotic manipulation and navigation tasks.
In reinforcement learning, an agent learns to reach a set of goals by means of an external reward signal. In the natural world, intelligent organisms learn from internal drives, bypassing the need for external signals, which is beneficial for a wide range of tasks. Motivated by this observation, we propose to formulate an intrinsic objective as the mutual information between the goal states and the controllable states. This objective encourages the agent to take control of its environment. Subsequently, we derive a surrogate objective of the proposed reward function, which can be optimized efficiently. Lastly, we evaluate the developed framework in different robotic manipulation and navigation tasks and demonstrate the efficacy of our approach. A video showing experimental results is available at https://youtu.be/CT4CKMWBYz0