Bridging Scene Understanding and Task Execution with Flexible Simulation Environments
This work provides a flexible simulation environment for researchers developing sophisticated perception and reinforcement learning algorithms, particularly for metric-semantic mapping and 3D dynamic scene graph generation.
This paper introduces TESSE, an open-source simulator designed to bridge scene understanding and task execution. It has been used to develop state-of-the-art solutions for metric-semantic mapping and 3D dynamic scene graph generation, and served as the platform for the GOSEEK Challenge at ICRA 2020.
Significant progress has been made in scene understanding which seeks to build 3D, metric and object-oriented representations of the world. Concurrently, reinforcement learning has made impressive strides largely enabled by advances in simulation. Comparatively, there has been less focus in simulation for perception algorithms. Simulation is becoming increasingly vital as sophisticated perception approaches such as metric-semantic mapping or 3D dynamic scene graph generation require precise 3D, 2D, and inertial information in an interactive environment. To that end, we present TESSE (Task Execution with Semantic Segmentation Environments), an open source simulator for developing scene understanding and task execution algorithms. TESSE has been used to develop state-of-the-art solutions for metric-semantic mapping and 3D dynamic scene graph generation. Additionally, TESSE served as the platform for the GOSEEK Challenge at the International Conference of Robotics and Automation (ICRA) 2020, an object search competition with an emphasis on reinforcement learning. Code for TESSE is available at https://github.com/MIT-TESSE.