Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoning
This addresses the layout generation bottleneck for 3D indoor scene synthesis, which is vital for embodied AI and digital content creation, though it appears incremental as it builds on existing LLM and spatial reasoning techniques.
The paper tackles the problem of generating realistic 3D indoor scene layouts from text descriptions, which is challenging due to limited datasets and trade-offs between flexibility and constraints. The result is DirectLayout, a framework that uses large language models with spatial reasoning to directly produce numerical layouts, achieving impressive semantic consistency, generalization, and physical plausibility.
Realistic 3D indoor scene synthesis is vital for embodied AI and digital content creation. It can be naturally divided into two subtasks: object generation and layout generation. While recent generative models have significantly advanced object-level quality and controllability, layout generation remains challenging due to limited datasets. Existing methods either overfit to these datasets or rely on predefined constraints to optimize numerical layout that sacrifice flexibility. As a result, they fail to generate scenes that are both open-vocabulary and aligned with fine-grained user instructions. We introduce DirectLayout, a framework that directly generates numerical 3D layouts from text descriptions using generalizable spatial reasoning of large language models (LLMs). DirectLayout decomposes the generation into three stages: producing a Bird's-Eye View (BEV) layout, lifting it into 3D space, and refining object placements. To enable explicit spatial reasoning and help the model grasp basic principles of object placement, we employ Chain-of-Thought (CoT) Activation based on the 3D-Front dataset. Additionally, we design CoT-Grounded Generative Layout Reward to enhance generalization and spatial planning. During inference, DirectLayout addresses asset-layout mismatches via Iterative Asset-Layout Alignment through in-context learning. Extensive experiments demonstrate that DirectLayout achieves impressive semantic consistency, generalization and physical plausibility.