RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation
This work addresses the challenge of realistic urban navigation for AI systems, providing a new dataset and baseline, but it is incremental as it builds on existing neural methods for navigation tasks.
The authors tackled the problem of interpreting natural language navigation instructions using dense urban maps by introducing the RUN dataset with 2515 instructions aligned with real routes in Manhattan, and they developed a baseline model showing that entity abstraction, attention mechanisms, and dynamic world-state updates significantly improve accuracy.
Following navigation instructions in natural language requires a composition of language, action, and knowledge of the environment. Knowledge of the environment may be provided via visual sensors or as a symbolic world representation referred to as a map. Here we introduce the Realistic Urban Navigation (RUN) task, aimed at interpreting navigation instructions based on a real, dense, urban map. Using Amazon Mechanical Turk, we collected a dataset of 2515 instructions aligned with actual routes over three regions of Manhattan. We propose a strong baseline for the task and empirically investigate which aspects of the neural architecture are important for the RUN success. Our results empirically show that entity abstraction, attention over words and worlds, and a constantly updating world-state, significantly contribute to task accuracy.