4Hammer: a board-game reinforcement learning environment for the hour long time frame
This addresses a gap in AI research for evaluating long-duration tasks, though it is incremental as it adapts an existing game to a new environment.
The authors tackled the lack of complex board game environments for reinforcement learning and LLM evaluation by proposing 4Hammer, a digital twin simulation of Warhammer 40,000, which requires understanding over 50 pages of rules and tracking evolving game states.
Large Language Models (LLMs) have demonstrated strong performance on tasks with short time frames, but struggle with tasks requiring longer durations. While datasets covering extended-duration tasks, such as software engineering tasks or video games, do exist, there are currently few implementations of complex board games specifically designed for reinforcement learning and LLM evaluation. To address this gap, we propose the 4Hammer reinforcement learning environment, a digital twin simulation of a subset of Warhammer 40,000-a complex, zero-sum board game. Warhammer 40,000 features intricate rules, requiring human players to thoroughly read and understand over 50 pages of detailed natural language rules, grasp the interactions between their game pieces and those of their opponents, and independently track and communicate the evolving game state.