Causal Reasoning Elicits Controllable 3D Scene Generation
This addresses the challenge of generating realistic and logically consistent 3D scenes for applications in virtual reality, gaming, and simulation, representing a novel method for a known bottleneck.
The paper tackled the problem of 3D scene generation methods struggling with logical dependencies and physical constraints, proposing CausalStruct, which embeds causal reasoning to improve scene layouts, resulting in enhanced logical coherence, realistic spatial interactions, and robust adaptability as shown in experiments.
Existing 3D scene generation methods often struggle to model the complex logical dependencies and physical constraints between objects, limiting their ability to adapt to dynamic and realistic environments. We propose CausalStruct, a novel framework that embeds causal reasoning into 3D scene generation. Utilizing large language models (LLMs), We construct causal graphs where nodes represent objects and attributes, while edges encode causal dependencies and physical constraints. CausalStruct iteratively refines the scene layout by enforcing causal order to determine the placement order of objects and applies causal intervention to adjust the spatial configuration according to physics-driven constraints, ensuring consistency with textual descriptions and real-world dynamics. The refined scene causal graph informs subsequent optimization steps, employing a Proportional-Integral-Derivative(PID) controller to iteratively tune object scales and positions. Our method uses text or images to guide object placement and layout in 3D scenes, with 3D Gaussian Splatting and Score Distillation Sampling improving shape accuracy and rendering stability. Extensive experiments show that CausalStruct generates 3D scenes with enhanced logical coherence, realistic spatial interactions, and robust adaptability.