Habitat-Matterport 3D Semantics Dataset
This dataset addresses the need for large-scale, high-quality 3D semantic annotations for the computer vision and robotics research community, enabling better training and evaluation of navigation models.
The authors introduced the Habitat-Matterport 3D Semantics (HM3DSEM) dataset, the largest 3D real-world dataset with dense semantic annotations, containing 142,646 object instances across 216 spaces and 3,100 rooms, and demonstrated its effectiveness by improving Object Goal Navigation performance and increasing challenge participation from 400 to 1022 submissions.
We present the Habitat-Matterport 3D Semantics (HM3DSEM) dataset. HM3DSEM is the largest dataset of 3D real-world spaces with densely annotated semantics that is currently available to the academic community. It consists of 142,646 object instance annotations across 216 3D spaces and 3,100 rooms within those spaces. The scale, quality, and diversity of object annotations far exceed those of prior datasets. A key difference setting apart HM3DSEM from other datasets is the use of texture information to annotate pixel-accurate object boundaries. We demonstrate the effectiveness of HM3DSEM dataset for the Object Goal Navigation task using different methods. Policies trained using HM3DSEM perform outperform those trained on prior datasets. Introduction of HM3DSEM in the Habitat ObjectNav Challenge lead to an increase in participation from 400 submissions in 2021 to 1022 submissions in 2022.