The Robotic Vision Scene Understanding Challenge
This provides a benchmark for researchers working on active robotic vision systems, though it is incremental as it builds on existing simulation and evaluation methods.
The authors tackled the lack of standardized testing for robotic scene understanding by introducing a new simulation-based challenge with tasks, difficulty levels, environments, and an evaluation measure for 3D object maps, aiming to drive state-of-the-art research.
Being able to explore an environment and understand the location and type of all objects therein is important for indoor robotic platforms that must interact closely with humans. However, it is difficult to evaluate progress in this area due to a lack of standardized testing which is limited due to the need for active robot agency and perfect object ground-truth. To help provide a standard for testing scene understanding systems, we present a new robot vision scene understanding challenge using simulation to enable repeatable experiments with active robot agency. We provide two challenging task types, three difficulty levels, five simulated environments and a new evaluation measure for evaluating 3D cuboid object maps. Our aim is to drive state-of-the-art research in scene understanding through enabling evaluation and comparison of active robotic vision systems.