Can LLMs Play Ô Ăn Quan Game? A Study of Multi-Step Planning and Decision Making
This study addresses the problem of assessing LLMs' multi-step planning and decision-making for researchers in AI and game theory, but it appears incremental as it applies existing methods to a new game environment without claiming major breakthroughs.
The paper tackled the problem of evaluating large language models' (LLMs) ability to plan and make strategic decisions by using the Vietnamese board game Ô Ăn Quan as a testbed, with results providing insights into their reasoning and strategic capabilities through experiments with models like Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, and Llama-3.3-70B-Instruct.
In this paper, we explore the ability of large language models (LLMs) to plan and make decisions through the lens of the traditional Vietnamese board game, Ô Ăn Quan. This game, which involves a series of strategic token movements and captures, offers a unique environment for evaluating the decision-making and strategic capabilities of LLMs. Specifically, we develop various agent personas, ranging from aggressive to defensive, and employ the Ô Ăn Quan game as a testbed for assessing LLM performance across different strategies. Through experimentation with models like Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, and Llama-3.3-70B-Instruct, we aim to understand how these models execute strategic decision-making, plan moves, and manage dynamic game states. The results will offer insights into the strengths and weaknesses of LLMs in terms of reasoning and strategy, contributing to a deeper understanding of their general capabilities.