CityX: Controllable Procedural Content Generation for Unbounded 3D Cities
This work addresses the need for high-quality 3D city generation to support embodied intelligence research, offering a practical solution for practitioners in fields such as robotics and autonomous systems.
The paper tackles the challenge of generating authentic, simulation-ready 3D cities for training and verifying embodied agents like autonomous vehicles, by developing CityX, a method that uses procedural content generation with a multi-agent framework to transform multi-modal instructions into executable programs, resulting in diverse, controllable, and realistic urban scenes that can be deployed as real-time simulators and infinite data generators.
Urban areas, as the primary human habitat in modern civilization, accommodate a broad spectrum of social activities. With the surge of embodied intelligence, recent years have witnessed an increasing presence of physical agents in urban areas, such as autonomous vehicles and delivery robots. As a result, practitioners significantly value crafting authentic, simulation-ready 3D cities to facilitate the training and verification of such agents. However, this task is quite challenging. Current generative methods fall short in either diversity, controllability, or fidelity. In this work, we resort to the procedural content generation (PCG) technique for high-fidelity generation. It assembles superior assets according to empirical rules, ultimately leading to industrial-grade outcomes. To ensure diverse and self contained creation, we design a management protocol to accommodate extensive PCG plugins with distinct functions and interfaces. Based on this unified PCG library, we develop a multi-agent framework to transform multi-modal instructions, including OSM, semantic maps, and satellite images, into executable programs. The programs coordinate relevant plugins to construct the 3D city consistent with the control condition. A visual feedback scheme is introduced to further refine the initial outcomes. Our method, named CityX, demonstrates its superiority in creating diverse, controllable, and realistic 3D urban scenes. The synthetic scenes can be seamlessly deployed as a real-time simulator and an infinite data generator for embodied intelligence research. Our project page: https://cityx-lab.github.io.