Crowd against the machine: A simulation-based benchmark tool to evaluate and compare robot capabilities to navigate a human crowd
This work addresses the need for standardized testing of robots in crowded environments, though it is incremental as it initiates tool development without major breakthroughs.
The paper tackles the problem of evaluating robot navigation in human crowds by developing a simulation-based benchmark tool, demonstrating its relevance through early results.
The evaluation of robot capabilities to navigate human crowds is essential to conceive new robots intended to operate in public spaces. This paper initiates the development of a benchmark tool to evaluate such capabilities; our long term vision is to provide the community with a simulation tool that generates virtual crowded environment to test robots, to establish standard scenarios and metrics to evaluate navigation techniques in terms of safety and efficiency, and thus, to install new methods to benchmarking robots' crowd navigation capabilities. This paper presents the architecture of the simulation tools, introduces first scenarios and evaluation metrics, as well as early results to demonstrate that our solution is relevant to be used as a benchmark tool.