Deep Reinforcement Learning Based Navigation with Macro Actions and Topological Maps
This addresses the problem of sample-efficient navigation for agents in complex environments, though it appears incremental as it builds on existing DQN methods with added abstractions.
The paper tackles navigation in large, visually complex environments with sparse rewards by using object-oriented macro actions and topological maps, showing that it significantly outperforms a random baseline in photorealistic 3D simulations.
This paper addresses the challenge of navigation in large, visually complex environments with sparse rewards. We propose a method that uses object-oriented macro actions grounded in a topological map, allowing a simple Deep Q-Network (DQN) to learn effective navigation policies. The agent builds a map by detecting objects from RGBD input and selecting discrete macro actions that correspond to navigating to these objects. This abstraction drastically reduces the complexity of the underlying reinforcement learning problem and enables generalization to unseen environments. We evaluate our approach in a photorealistic 3D simulation and show that it significantly outperforms a random baseline under both immediate and terminal reward conditions. Our results demonstrate that topological structure and macro-level abstraction can enable sample-efficient learning even from pixel data.