Grammar and Gameplay-aligned RL for Game Description Generation with LLMs
This work addresses the problem of generating accurate and grammatically correct game descriptions for game developers or AI researchers, representing an incremental improvement over existing LLM-based methods.
The paper tackles the challenge of accurately generating game descriptions in a Game Description Language from natural language by proposing RLGDG, a reinforcement learning-based fine-tuning method for LLMs that improves grammatical correctness and fidelity to game concepts, significantly outperforming baseline methods using supervised fine-tuning alone.
Game Description Generation (GDG) is the task of generating a game description written in a Game Description Language (GDL) from natural language text. Previous studies have explored generation methods leveraging the contextual understanding capabilities of Large Language Models (LLMs); however, accurately reproducing the game features of the game descriptions remains a challenge. In this paper, we propose reinforcement learning-based fine-tuning of LLMs for GDG (RLGDG). Our training method simultaneously improves grammatical correctness and fidelity to game concepts by introducing both grammar rewards and concept rewards. Furthermore, we adopt a two-stage training strategy where Reinforcement Learning (RL) is applied following Supervised Fine-Tuning (SFT). Experimental results demonstrate that our proposed method significantly outperforms baseline methods using SFT alone. Our code is available at https://github.com/tsunehiko/rlgdg