Does ChatGPT have Theory of Mind?
This addresses the problem of assessing AI's social cognition capabilities for researchers and developers, though it is incremental in evaluating existing models.
The paper investigates whether ChatGPT models possess Theory of Mind by testing them on six problems related to human reasoning biases, finding that ChatGPT-4 achieves correct answers more often than chance but often based on flawed reasoning.
Theory of Mind (ToM) is the ability to understand human thinking and decision-making, an ability that plays a crucial role in social interaction between people, including linguistic communication. This paper investigates to what extent recent Large Language Models in the ChatGPT tradition possess ToM. We posed six well-known problems that address biases in human reasoning and decision making to two versions of ChatGPT and we compared the results under a range of prompting strategies. While the results concerning ChatGPT-3 were somewhat inconclusive, ChatGPT-4 was shown to arrive at the correct answers more often than would be expected based on chance, although correct answers were often arrived at on the basis of false assumptions or invalid reasoning.