Mastering Da Vinci Code: A Comparative Study of Transformer, LLM, and PPO-based Agents
This study addresses the problem of designing effective AI agents for recreational games with hidden information and multi-step reasoning, offering insights into agent design and comparative AI approaches, but it is incremental as it applies existing methods to a new domain.
The paper tackled the problem of mastering the Da Vinci Code game, a logical deduction challenge with imperfect information, by comparing Transformer, LLM, and PPO-based agents, with the PPO-based agent achieving a superior win rate of 58.5% ± 1.0%.
The Da Vinci Code, a game of logical deduction and imperfect information, presents unique challenges for artificial intelligence, demanding nuanced reasoning beyond simple pattern recognition. This paper investigates the efficacy of various AI paradigms in mastering this game. We develop and evaluate three distinct agent architectures: a Transformer-based baseline model with limited historical context, several Large Language Model (LLM) agents (including Gemini, DeepSeek, and GPT variants) guided by structured prompts, and an agent based on Proximal Policy Optimization (PPO) employing a Transformer encoder for comprehensive game history processing. Performance is benchmarked against the baseline, with the PPO-based agent demonstrating superior win rates ($58.5\% \pm 1.0\%$), significantly outperforming the LLM counterparts. Our analysis highlights the strengths of deep reinforcement learning in policy refinement for complex deductive tasks, particularly in learning implicit strategies from self-play. We also examine the capabilities and inherent limitations of current LLMs in maintaining strict logical consistency and strategic depth over extended gameplay, despite sophisticated prompting. This study contributes to the broader understanding of AI in recreational games involving hidden information and multi-step logical reasoning, offering insights into effective agent design and the comparative advantages of different AI approaches.