ToLL: Topological Layout Learning with Structural Multi-view Augmentation for 3D Scene Graph Pretraining
This work addresses a domain-specific challenge in 3D spatial understanding for applications like robotics or AR/VR, but it is incremental as it builds on existing methods for 3DSG pretraining.
The paper tackles the problem of data scarcity in 3D Scene Graph generation by proposing ToLL, a self-supervised pretraining framework that improves representation quality and outperforms state-of-the-art baselines on the 3DSSG dataset.
3D Scene Graph (3DSG) generation plays a pivotal role in spatial understanding and semantic-affordance perception. However, its generalizability is often constrained by data scarcity. Current solutions primarily focus on cross-modal assisted representation learning and object-centric generation pre-training. The former relies heavily on predicate annotations, while the latter's predicate learning may be bypassed due to strong object priors. Consequently, they could not often provide a label-free and robust self-supervised proxy task for 3DSG fine-tuning. To bridge this gap, we propose a Topological Layout Learning (ToLL) for 3DSG pretraining framework. In detail, we design an Anchor-Conditioned Topological Geometry Reasoning, with a GNN to recover the global layout of zero-centered subgraphs by the spatial priors from sparse anchors. This process is strictly modulated by predicate features, thereby enforcing the predicate relation learning. Furthermore, we construct a Structural Multi-view Augmentation to avoid semantic corruption, and enhancing representations via self-distillation. The extensive experiments on 3DSSG dataset demonstrate that our ToLL could improve representation quality, outperforming state-of-the-art baselines.