Benchmarking Classic and Learned Navigation in Complex 3D Environments
This work addresses the disconnect between classic and learning-based navigation research for robotics and AI applications, providing a benchmarking study.
The paper tackled the problem of comparing classic and learning-based navigation methods in complex 3D environments, finding that a tuned classic pipeline performs well in clutter while learned systems are more robust with limited sensors, but both are far from human-level performance.
Navigation research is attracting renewed interest with the advent of learning-based methods. However, this new line of work is largely disconnected from well-established classic navigation approaches. In this paper, we take a step towards coordinating these two directions of research. We set up classic and learning-based navigation systems in common simulated environments and thoroughly evaluate them in indoor spaces of varying complexity, with access to different sensory modalities. Additionally, we measure human performance in the same environments. We find that a classic pipeline, when properly tuned, can perform very well in complex cluttered environments. On the other hand, learned systems can operate more robustly with a limited sensor suite. Overall, both approaches are still far from human-level performance.