SceneX: Procedural Controllable Large-scale Scene Generation
This work addresses challenges in scalable scene generation for designers and developers, offering an incremental improvement by automating procedural model creation.
The paper tackles the problem of limited modular diversity and high expertise requirements in procedural controllable generation (PCG) for large-scale scenes by introducing SceneX, a framework that automatically produces high-quality procedural models from textual descriptions, demonstrated through experiments on nature scenes and unbounded cities.
Developing comprehensive explicit world models is crucial for understanding and simulating real-world scenarios. Recently, Procedural Controllable Generation (PCG) has gained significant attention in large-scale scene generation by enabling the creation of scalable, high-quality assets. However, PCG faces challenges such as limited modular diversity, high expertise requirements, and challenges in managing the diverse elements and structures in complex scenes. In this paper, we introduce a large-scale scene generation framework, SceneX, which can automatically produce high-quality procedural models according to designers' textual descriptions. Specifically, the proposed method comprises two components, PCGHub and PCGPlanner. The former encompasses an extensive collection of accessible procedural assets and thousands of hand-craft API documents to perform as a standard protocol for PCG controller. The latter aims to generate executable actions for Blender to produce controllable and precise 3D assets guided by the user's instructions. Extensive experiments demonstrated the capability of our method in controllable large-scale scene generation, including nature scenes and unbounded cities, as well as scene editing such as asset placement and season translation.