SidewalkBench: Benchmarking Visual Navigation on Urban Sidewalks
Provides the first comprehensive benchmark for sidewalk navigation, enabling reproducible evaluation for researchers in visual navigation and robotics.
SidewalkBench introduces a unified benchmark for visual navigation on urban sidewalks, enabling standardized evaluation of 9 models across 330 unit-test, 800 pedestrian-reactive, and 105 long-horizon scenarios. Results show pedestrian interaction and long-horizon robustness are critical bottlenecks, with synthetic data scaling as a promising solution.
Urban sidewalk navigation presents significant challenges due to complex structural layouts, dynamic pedestrian behaviors, and long distances. While recent visual navigation models offer a promising solution, the lack of a unified benchmark hinders quantitative and reproducible evaluation. To bridge this gap, we propose SidewalkBench, a comprehensive benchmark designed for visual navigation on urban sidewalks. Built upon NVIDIA Isaac Sim, SidewalkBench brings GPU-accelerated simulation of diverse, high-fidelity sidewalk environments, including both procedurally generated and real-world scanned scenes. We further populate the scenes with rich, reactive event-based pedestrian behaviors and flexible, efficient animation, enabling standardized model evaluation under realistic real-world settings. We conduct a comprehensive evaluation of 9 visual navigation models on 330 unit-test scenarios, 800 pedestrian-reactive scenarios, and 105 long-horizon scenarios. Our findings highlight that pedestrian interaction and long-horizon robustness remain critical bottlenecks for existing models, and scaling up sidewalk training with synthetic data emerges as a promising solution.