TerraFusion: Joint Generation of Terrain Geometry and Texture Using Latent Diffusion Models
This addresses the need for more realistic terrain generation in video games and films, though it is incremental as it builds on existing diffusion models.
The paper tackles the problem of generating realistic 3D terrain models by jointly producing heightmaps and textures using a latent diffusion model, with experiments demonstrating intuitive user control via sketches while preserving geometry-texture correlation.
3D terrain models are essential in fields such as video game development and film production. Since surface color often correlates with terrain geometry, capturing this relationship is crucial to achieving realism. However, most existing methods generate either a heightmap or a texture, without sufficiently accounting for the inherent correlation. In this paper, we propose a method that jointly generates terrain heightmaps and textures using a latent diffusion model. First, we train the model in an unsupervised manner to randomly generate paired heightmaps and textures. Then, we perform supervised learning of an external adapter to enable user control via hand-drawn sketches. Experiments show that our approach allows intuitive terrain generation while preserving the correlation between heightmaps and textures.