Metacognition and Uncertainty Communication in Humans and Large Language Models
This work addresses the problem of assessing and enhancing metacognition in LLMs for better integration into high-stakes and everyday applications, though it is incremental as it reviews and synthesizes existing knowledge.
The paper examines the metacognitive abilities of large language models (LLMs) compared to humans, finding that while there are some alignments, significant differences remain, which are crucial for improving human-AI collaboration.
Metacognition--the capacity to monitor and evaluate one's own knowledge and performance--is foundational to human decision-making, learning, and communication. As large language models (LLMs) become increasingly embedded in both high-stakes and widespread low-stakes contexts, it is important to assess whether, how, and to what extent they exhibit metacognitive abilities. Here, we provide an overview of current knowledge of LLMs' metacognitive capacities, how they might be studied, and how they relate to our knowledge of metacognition in humans. We show that while humans and LLMs can sometimes appear quite aligned in their metacognitive capacities and behaviors, it is clear many differences remain; attending to these differences is important for enhancing human-AI collaboration. Finally, we discuss how endowing future LLMs with more sensitive and more calibrated metacognition may also help them develop new capacities such as more efficient learning, self-direction, and curiosity.