An Autonomous Non-monolithic Agent with Multi-mode Exploration based on Options Framework
This work addresses the challenge of when to explore in RL, which is less studied than how to explore, potentially improving efficiency for RL agents in tasks requiring adaptive behavior.
The paper tackles the problem of enabling autonomous decision-making between exploration and exploitation in reinforcement learning by introducing a non-monolithic, multi-mode exploration method based on an options framework, showing higher performance compared to existing non-monolithic methods in experiments.
Most exploration research on reinforcement learning (RL) has paid attention to `the way of exploration', which is `how to explore'. The other exploration research, `when to explore', has not been the main focus of RL exploration research. The issue of `when' of a monolithic exploration in the usual RL exploration behaviour binds an exploratory action to an exploitational action of an agent. Recently, a non-monolithic exploration research has emerged to examine the mode-switching exploration behaviour of humans and animals. The ultimate purpose of our research is to enable an agent to decide when to explore or exploit autonomously. We describe the initial research of an autonomous multi-mode exploration of non-monolithic behaviour in an options framework. The higher performance of our method is shown against the existing non-monolithic exploration method through comparative experimental results.