Limits of Emergent Reasoning of Large Language Models in Agentic Frameworks for Deterministic Games
This addresses the issue of evaluating true reasoning in LLMs for researchers, showing incremental insights into their limitations in agentic frameworks.
The study tackled the problem of whether providing an environment interface to large language models (LLMs) can prevent performance collapse in solving deterministic games like Tower of Hanoi, finding that it does not delay or eradicate the collapse, with models diverging from optimal policies as complexity increases.
Recent work reports that Large Reasoning Models (LRMs) undergo a collapse in performance on solving puzzles beyond certain perplexity thresholds. In subsequent discourse, questions have arisen as to whether the nature of the task muddles an evaluation of true reasoning. One potential confound is the requirement that the model keep track of the state space on its own. We provide a large language model (LLM) with an environment interface for Tower of Hanoi problems, allowing it to make a move with a tool call, provide written justification, observe the resulting state space, and reprompt itself for the next move. We observe that access to an environment interface does not delay or eradicate performance collapse. Furthermore, LLM-parameterized policy analysis reveals increasing divergence from both optimal policies and uniformly random policies, suggesting that the model exhibits mode-like collapse at each level of complexity, and that performance is dependent upon whether the mode reflects the correct solution for the problem. We suggest that a similar phenomena might take place in LRMs.