R3DS: Reality-linked 3D Scenes for Panoramic Scene Understanding
This provides a new dataset for researchers in computer vision to benchmark and advance panoramic scene understanding, though it is incremental as it builds on existing datasets like Matterport3D.
The paper tackles the problem of panoramic scene understanding by introducing the R3DS dataset, which contains 19K objects from 3,784 CAD models across 100 categories, and shows that training on it improves generalization and support relation prediction compared to heuristic methods.
We introduce the Reality-linked 3D Scenes (R3DS) dataset of synthetic 3D scenes mirroring the real-world scene arrangements from Matterport3D panoramas. Compared to prior work, R3DS has more complete and densely populated scenes with objects linked to real-world observations in panoramas. R3DS also provides an object support hierarchy, and matching object sets (e.g., same chairs around a dining table) for each scene. Overall, R3DS contains 19K objects represented by 3,784 distinct CAD models from over 100 object categories. We demonstrate the effectiveness of R3DS on the Panoramic Scene Understanding task. We find that: 1) training on R3DS enables better generalization; 2) support relation prediction trained with R3DS improves performance compared to heuristically calculated support; and 3) R3DS offers a challenging benchmark for future work on panoramic scene understanding.