Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models
This work addresses the problem of reliably assessing AI social reasoning for researchers and practitioners, cautioning against overinterpretation of anecdotal evidence, and is incremental in refining evaluation metrics.
The paper investigates whether large language models (LLMs) like ChatGPT and GPT-4 exhibit robust Neural Theory-of-Mind (N-ToM) abilities, finding through evaluation on 6 tasks that their performance is not robust and relies on shallow heuristics, especially under adversarial conditions.
The escalating debate on AI's capabilities warrants developing reliable metrics to assess machine "intelligence". Recently, many anecdotal examples were used to suggest that newer large language models (LLMs) like ChatGPT and GPT-4 exhibit Neural Theory-of-Mind (N-ToM); however, prior work reached conflicting conclusions regarding those abilities. We investigate the extent of LLMs' N-ToM through an extensive evaluation on 6 tasks and find that while LLMs exhibit certain N-ToM abilities, this behavior is far from being robust. We further examine the factors impacting performance on N-ToM tasks and discover that LLMs struggle with adversarial examples, indicating reliance on shallow heuristics rather than robust ToM abilities. We caution against drawing conclusions from anecdotal examples, limited benchmark testing, and using human-designed psychological tests to evaluate models.