Violation of Expectation via Metacognitive Prompting Reduces Theory of Mind Prediction Error in Large Language Models
This addresses the challenge of improving AI-human interactions, particularly in AI tutoring, though it appears incremental as it builds on existing LLM Theory of Mind capabilities.
The paper tackles the problem of reducing Theory of Mind prediction errors in Large Language Models by implementing a Violation of Expectation mechanism through metacognitive prompting, resulting in LLMs learning about users in ways that echo human learning theories.
Recent research shows that Large Language Models (LLMs) exhibit a compelling level of proficiency in Theory of Mind (ToM) tasks. This ability to impute unobservable mental states to others is vital to human social cognition and may prove equally important in principal-agent relations between individual humans and Artificial Intelligences (AIs). In this paper, we explore how a mechanism studied in developmental psychology known as Violation of Expectation (VoE) can be implemented to reduce errors in LLM prediction about users by leveraging emergent ToM affordances. And we introduce a \textit{metacognitive prompting} framework to apply VoE in the context of an AI tutor. By storing and retrieving facts derived in cases where LLM expectation about the user was violated, we find that LLMs are able to learn about users in ways that echo theories of human learning. Finally, we discuss latent hazards and augmentative opportunities associated with modeling user psychology and propose ways to mitigate risk along with possible directions for future inquiry.