Pair2Scene: Learning Local Object Relations for Procedural Scene Generation
For 3D scene generation, Pair2Scene addresses the bottleneck of scaling to dense scenes with precise spatial reasoning, offering a procedural approach that outperforms LLM/VLM-based methods.
Pair2Scene generates high-fidelity 3D indoor scenes by learning local object relations (support and functional) from a curated dataset 3D-Pairs, and using hierarchical recursive generation with collision-aware sampling. It outperforms existing methods in generating complex, physically and semantically plausible scenes beyond training data.
Generating high-fidelity 3D indoor scenes remains a significant challenge due to data scarcity and the complexity of modeling intricate spatial relations. Current methods often struggle to scale beyond training distribution to dense scenes or rely on LLMs/VLMs that lack the ability for precise spatial reasoning. Building on top of the observation that object placement relies mainly on local dependencies instead of information-redundant global distributions, in this paper, we propose Pair2Scene, a novel procedural generation framework that integrates learned local rules with scene hierarchies and physics-based algorithms. These rules mainly capture two types of inter-object relations, namely support relations that follow physical hierarchies, and functional relations that reflect semantic links. We model these rules through a network, which estimates spatial position distributions of dependent objects conditioned on position and geometry of the anchor ones. Accordingly, we curate a dataset 3D-Pairs from existing scene data to train the model. During inference, our framework can generate scenes by recursively applying our model within a hierarchical structure, leveraging collision-aware rejection sampling to align local rules into coherent global layouts. Extensive experiments demonstrate that our framework outperforms existing methods in generating complex environments that go beyond training data while maintaining physical and semantic plausibility.