ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind
This work addresses the problem of evaluating Theory of Mind in AI systems for researchers and developers, but it is incremental as it builds on existing benchmark approaches with enhancements in scope and diversity.
The authors tackled the limitations of existing Theory of Mind benchmarks by introducing ToMATO, a new benchmark with 5.4k questions and 753 conversations that assesses a broader range of mental states and includes false beliefs and personality traits, finding that even advanced LLMs like GPT-4o mini lag behind humans, particularly in false belief understanding.
Existing Theory of Mind (ToM) benchmarks diverge from real-world scenarios in three aspects: 1) they assess a limited range of mental states such as beliefs, 2) false beliefs are not comprehensively explored, and 3) the diverse personality traits of characters are overlooked. To address these challenges, we introduce ToMATO, a new ToM benchmark formulated as multiple-choice QA over conversations. ToMATO is generated via LLM-LLM conversations featuring information asymmetry. By employing a prompting method that requires role-playing LLMs to verbalize their thoughts before each utterance, we capture both first- and second-order mental states across five categories: belief, intention, desire, emotion, and knowledge. These verbalized thoughts serve as answers to questions designed to assess the mental states of characters within conversations. Furthermore, the information asymmetry introduced by hiding thoughts from others induces the generation of false beliefs about various mental states. Assigning distinct personality traits to LLMs further diversifies both utterances and thoughts. ToMATO consists of 5.4k questions, 753 conversations, and 15 personality trait patterns. Our analysis shows that this dataset construction approach frequently generates false beliefs due to the information asymmetry between role-playing LLMs, and effectively reflects diverse personalities. We evaluate nine LLMs on ToMATO and find that even GPT-4o mini lags behind human performance, especially in understanding false beliefs, and lacks robustness to various personality traits.