3D Building Generation in Minecraft via Large Language Models
This work addresses procedural content generation for 3D game environments, specifically for Minecraft players and developers, but it is incremental as it extends existing LLM methods from 2D to 3D generation.
The paper tackled 3D building generation in Minecraft using large language models (LLMs), proposing a T2BM model that supports facade, indoor scenes, and functional blocks, and found that LLMs can generate correct buildings with complete structures and specific blocks like windows and beds when given appropriate prompts.
Recently, procedural content generation has exhibited considerable advancements in the domain of 2D game level generation such as Super Mario Bros. and Sokoban through large language models (LLMs). To further validate the capabilities of LLMs, this paper explores how LLMs contribute to the generation of 3D buildings in a sandbox game, Minecraft. We propose a Text to Building in Minecraft (T2BM) model, which involves refining prompts, decoding interlayer representation and repairing. Facade, indoor scene and functional blocks like doors are supported in the generation. Experiments are conducted to evaluate the completeness and satisfaction of buildings generated via LLMs. It shows that LLMs hold significant potential for 3D building generation. Given appropriate prompts, LLMs can generate correct buildings in Minecraft with complete structures and incorporate specific building blocks such as windows and beds, meeting the specified requirements of human users.