B2N3D: Progressive Learning from Binary to N-ary Relationships for 3D Object Grounding
This work addresses the challenge of 3D object grounding for robotic scene understanding, which is incremental by extending existing relational learning from binary to n-ary relationships.
The paper tackles the problem of localizing 3D objects using natural language by addressing the limitation of current methods that only model pairwise relationships, proposing a progressive relational learning framework that extends to n-ary relationships to improve global perceptual understanding. It demonstrates state-of-the-art performance on benchmarks like ReferIt3D and ScanRefer, showing advantages in 3D localization.
Localizing 3D objects using natural language is essential for robotic scene understanding. The descriptions often involve multiple spatial relationships to distinguish similar objects, making 3D-language alignment difficult. Current methods only model relationships for pairwise objects, ignoring the global perceptual significance of n-ary combinations in multi-modal relational understanding. To address this, we propose a novel progressive relational learning framework for 3D object grounding. We extend relational learning from binary to n-ary to identify visual relations that match the referential description globally. Given the absence of specific annotations for referred objects in the training data, we design a grouped supervision loss to facilitate n-ary relational learning. In the scene graph created with n-ary relationships, we use a multi-modal network with hybrid attention mechanisms to further localize the target within the n-ary combinations. Experiments and ablation studies on the ReferIt3D and ScanRefer benchmarks demonstrate that our method outperforms the state-of-the-art, and proves the advantages of the n-ary relational perception in 3D localization.