Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning
This is an incremental position paper critiquing evaluation methods for ToM in AI, relevant for researchers in cognitive science and LLM benchmarking.
The paper argues that current AI research on Theory of Mind (ToM) in LLMs overly focuses on static inference tasks, neglecting the initial step of determining when and at what depth to invoke ToM, and suggests improved evaluation using dynamic cognitive environments.
Theory of Mind (ToM) capabilities in LLMs have recently become a central object of investigation. Cognitive science distinguishes between two steps required for ToM tasks: 1) determine whether to invoke ToM, which includes the appropriate Depth of Mentalizing (DoM), or level of recursion required to complete a task; and 2) applying the correct inference given the DoM. In this position paper, we first identify several lines of work in different communities in AI, including LLM benchmarking, ToM add-ons, ToM probing, and formal models for ToM. We argue that recent work in AI tends to focus exclusively on the second step which are typically framed as static logic problems. We conclude with suggestions for improved evaluation of ToM capabilities inspired by dynamic environments used in cognitive tasks.