GameUIAgent: An LLM-Powered Framework for Automated Game UI Design with Structured Intermediate Representation
This addresses the manual and time-consuming process of game UI design for developers, though it appears incremental as it builds on existing LLM and VLM techniques.
The authors tackled the problem of automating game UI design by developing GameUIAgent, an LLM-powered framework that converts natural language descriptions into editable Figma designs, achieving evaluation across 110 test cases and uncovering key empirical findings like a Quality Ceiling Effect with Pearson r=-0.96.
Game UI design requires consistent visual assets across rarity tiers yet remains a predominantly manual process. We present GameUIAgent, an LLM-powered agentic framework that translates natural language descriptions into editable Figma designs via a Design Spec JSON intermediate representation. A six-stage neuro-symbolic pipeline combines LLM generation, deterministic post-processing, and a Vision-Language Model (VLM)-guided Reflection Controller (RC) for iterative self-correction with guaranteed non-regressive quality. Evaluated across 110 test cases, three LLMs, and three UI templates, cross-model analysis establishes a game-domain failure taxonomy (rarity-dependent degradation; visual emptiness) and uncovers two key empirical findings. A Quality Ceiling Effect (Pearson r=-0.96, p<0.01) suggests that RC improvement is bounded by headroom below a quality threshold -- a visual-domain counterpart to test-time compute scaling laws. A Rendering-Evaluation Fidelity Principle reveals that partial rendering enhancements paradoxically degrade VLM evaluation by amplifying structural defects. Together, these results establish foundational principles for LLM-driven visual generation agents in game production.