CoPa-SG: Dense Scene Graphs with Parametric and Proto-Relations
This work addresses a data bottleneck in scene understanding for computer vision researchers, offering incremental improvements through new relation types and a synthetic dataset.
The paper tackles the lack of accurate scene graph data by introducing CoPa-SG, a synthetic dataset with precise ground truth and exhaustive relation annotations, and proposes parametric and proto-relations to enhance scene graph representation and downstream applications.
2D scene graphs provide a structural and explainable framework for scene understanding. However, current work still struggles with the lack of accurate scene graph data. To overcome this data bottleneck, we present CoPa-SG, a synthetic scene graph dataset with highly precise ground truth and exhaustive relation annotations between all objects. Moreover, we introduce parametric and proto-relations, two new fundamental concepts for scene graphs. The former provides a much more fine-grained representation than its traditional counterpart by enriching relations with additional parameters such as angles or distances. The latter encodes hypothetical relations in a scene graph and describes how relations would form if new objects are placed in the scene. Using CoPa-SG, we compare the performance of various scene graph generation models. We demonstrate how our new relation types can be integrated in downstream applications to enhance planning and reasoning capabilities.