CC3D: Layout-Conditioned Generation of Compositional 3D Scenes
This addresses the problem of controllable 3D scene generation for applications in computer graphics, VR/AR, and autonomous systems, representing a novel method rather than an incremental improvement.
The paper tackles the problem of generating complex 3D scenes with multiple objects by introducing CC3D, a conditional generative model that synthesizes 3D scenes from 2D semantic layouts using single-view images. The result is improved visual and geometric quality compared to previous works, as demonstrated on synthetic 3D-FRONT and real-world KITTI-360 datasets.
In this work, we introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts, trained using single-view images. Different from most existing 3D GANs that limit their applicability to aligned single objects, we focus on generating complex scenes with multiple objects, by modeling the compositional nature of 3D scenes. By devising a 2D layout-based approach for 3D synthesis and implementing a new 3D field representation with a stronger geometric inductive bias, we have created a 3D GAN that is both efficient and of high quality, while allowing for a more controllable generation process. Our evaluations on synthetic 3D-FRONT and real-world KITTI-360 datasets demonstrate that our model generates scenes of improved visual and geometric quality in comparison to previous works.