Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs
This work addresses the challenge of efficient real-time mapping for mobile robots in unknown environments, representing an incremental improvement over existing belief space planning methods.
The paper tackles the problem of autonomous exploration for mapping unknown environments by addressing the scalability and real-time limitations of belief space planning. The proposed method uses graph neural networks with deep reinforcement learning to achieve real-time, high-performance exploration, resulting in accurate maps and high information gain rates.
We consider an autonomous exploration problem in which a range-sensing mobile robot is tasked with accurately mapping the landmarks in an a priori unknown environment efficiently in real-time; it must choose sensing actions that both curb localization uncertainty and achieve information gain. For this problem, belief space planning methods that forward-simulate robot sensing and estimation may often fail in real-time implementation, scaling poorly with increasing size of the state, belief and action spaces. We propose a novel approach that uses graph neural networks (GNNs) in conjunction with deep reinforcement learning (DRL), enabling decision-making over graphs containing exploration information to predict a robot's optimal sensing action in belief space. The policy, which is trained in different random environments without human intervention, offers a real-time, scalable decision-making process whose high-performance exploratory sensing actions yield accurate maps and high rates of information gain.