NavTrust: Benchmarking Trustworthiness for Embodied Navigation
This addresses the need for more trustworthy embodied navigation systems by exposing robustness issues in real-world scenarios, though it is incremental as it builds on existing benchmarks and models.
The authors tackled the problem that embodied navigation models are typically evaluated only under ideal conditions, overlooking real-world corruptions, by creating NavTrust, a unified benchmark that systematically corrupts RGB, depth, and instruction inputs. Their evaluation of seven state-of-the-art models showed substantial performance degradation under these corruptions, highlighting critical robustness gaps.
There are two major categories of embodied navigation: Vision-Language Navigation (VLN), where agents navigate by following natural language instructions; and Object-Goal Navigation (OGN), where agents navigate to a specified target object. However, existing work primarily evaluates model performance under nominal conditions, overlooking the potential corruptions that arise in real-world settings. To address this gap, we present NavTrust, a unified benchmark that systematically corrupts input modalities, including RGB, depth, and instructions, in realistic scenarios and evaluates their impact on navigation performance. To our best knowledge, NavTrust is the first benchmark that exposes embodied navigation agents to diverse RGB-Depth corruptions and instruction variations in a unified framework. Our extensive evaluation of seven state-of-the-art approaches reveals substantial performance degradation under realistic corruptions, which highlights critical robustness gaps and provides a roadmap toward more trustworthy embodied navigation systems. Furthermore, we systematically evaluate four distinct mitigation strategies to enhance robustness against RGB-Depth and instructions corruptions. Our base models include Uni-NaVid and ETPNav. We deployed them on a real mobile robot and observed improved robustness to corruptions. The project website is: https://navtrust.github.io.