TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments
This addresses the challenge of efficient exploration and navigation for autonomous robots in sparse-reward settings, representing a strong specific gain rather than a foundational advancement.
The paper tackles the problem of autonomous robot navigation in unknown, sparse-reward environments by introducing TopoNav, a topological navigation framework that integrates active mapping, hierarchical reinforcement learning, and intrinsic motivation. The results show improvements in exploration coverage by 7-20%, success rates by 9-19%, and reductions in navigation times by 15-36% compared to state-of-the-art methods.
Autonomous robots exploring unknown environments face a significant challenge: navigating effectively without prior maps and with limited external feedback. This challenge intensifies in sparse reward environments, where traditional exploration techniques often fail. In this paper, we present TopoNav, a novel topological navigation framework that integrates active mapping, hierarchical reinforcement learning, and intrinsic motivation to enable efficient goal-oriented exploration and navigation in sparse-reward settings. TopoNav dynamically constructs a topological map of the environment, capturing key locations and pathways. A two-level hierarchical policy architecture, comprising a high-level graph traversal policy and low-level motion control policies, enables effective navigation and obstacle avoidance while maintaining focus on the overall goal. Additionally, TopoNav incorporates intrinsic motivation to guide exploration toward relevant regions and frontier nodes in the topological map, addressing the challenges of sparse extrinsic rewards. We evaluate TopoNav both in the simulated and real-world off-road environments using a Clearpath Jackal robot, across three challenging navigation scenarios: goal-reaching, feature-based navigation, and navigation in complex terrains. We observe an increase in exploration coverage by 7- 20%, in success rates by 9-19%, and reductions in navigation times by 15-36% across various scenarios, compared to state-of-the-art methods