Do Persona-Infused LLMs Affect Performance in a Strategic Reasoning Game?
This work addresses how persona prompting influences LLM-based decision-making in adversarial environments, proposing a heuristic generation method with psychometric principles, but it is incremental as it builds on existing persona and strategic reasoning research.
The study investigated whether persona prompting in large language models affects strategic decision-making in a board game called PERIL, finding that certain personas improved game performance only when a mediator translated them into heuristics, enhancing reliability and validity compared to direct inference.
Although persona prompting in large language models appears to trigger different styles of generated text, it is unclear whether these translate into measurable behavioral differences, much less whether they affect decision-making in an adversarial strategic environment that we provide as open-source. We investigate the impact of persona prompting on strategic performance in PERIL, a world-domination board game. Specifically, we compare the effectiveness of persona-derived heuristic strategies to those chosen manually. Our findings reveal that certain personas associated with strategic thinking improve game performance, but only when a mediator is used to translate personas into heuristic values. We introduce this mediator as a structured translation process, inspired by exploratory factor analysis, that maps LLM-generated inventory responses into heuristics. Results indicate our method enhances heuristic reliability and face validity compared to directly inferred heuristics, allowing us to better study the effect of persona types on decision making. These insights advance our understanding of how persona prompting influences LLM-based decision-making and propose a heuristic generation method that applies psychometric principles to LLMs.