Game of Thought: Robust Information Seeking with Large Language Models Using Game Theory
This addresses a critical issue for high-stakes applications where robust information-seeking is needed, though it is incremental as it builds on existing game theory and LLM techniques.
The paper tackled the problem of large language models lacking sufficient information for tasks by proposing a game-theoretic framework to improve worst-case performance in adversarial information-seeking scenarios, achieving consistent improvements over baseline methods.
Large Language Models (LLMs) are increasingly deployed in real-world scenarios where they may lack sufficient information to complete a given task. In such settings, the ability to actively seek out missing information becomes a critical capability. Existing approaches to enhancing this ability often rely on simplifying assumptions that degrade \textit{worst-case} performance. This is an issue with serious implications in high-stakes applications. In this work, we use the game of Twenty Questions to evaluate the information-seeking ability of LLMs. We introduce and formalize its adversarial counterpart, the Strategic Language Search (SLS) problem along with its variants as a two-player zero-sum extensive form game. We propose Game of Thought (GoT), a framework that applies game-theoretic techniques to approximate a Nash equilibrium (NE) strategy for the restricted variant of the game. Empirical results demonstrate that our approach consistently improves worst-case performance compared to (1) direct prompting-based methods and (2) heuristic-guided search methods across all tested settings.