How Different AI Chatbots Behave? Benchmarking Large Language Models in Behavioral Economics Games
This work addresses the need to understand AI chatbot behavior for deployment in critical decision-making roles, but it is incremental as it supplements a prior study on behavioral Turing tests.
The paper analyzed five leading LLM-based chatbots in behavioral economics games to understand their decision-making strategies, revealing common and distinct behavioral patterns that provide insights into their strategic preferences.
The deployment of large language models (LLMs) in diverse applications requires a thorough understanding of their decision-making strategies and behavioral patterns. As a supplement to a recent study on the behavioral Turing test, this paper presents a comprehensive analysis of five leading LLM-based chatbot families as they navigate a series of behavioral economics games. By benchmarking these AI chatbots, we aim to uncover and document both common and distinct behavioral patterns across a range of scenarios. The findings provide valuable insights into the strategic preferences of each LLM, highlighting potential implications for their deployment in critical decision-making roles.