Competence-Aware AI Agents with Metacognition for Unknown Situations and Environments (MUSE)
This work addresses the challenge of adaptability in unknown situations for autonomous agents, offering a novel approach that could reduce reliance on extensive training data, though it appears incremental as it builds on existing concepts like world modeling and LLMs.
The authors tackled the problem of autonomous agents struggling in novel environments by proposing the MUSE framework, which integrates metacognitive processes like competence awareness and strategy selection, resulting in agents that demonstrate high competence awareness and significant improvements in solving out-of-distribution tasks compared to existing methods.
Metacognition, defined as the awareness and regulation of one's cognitive processes, is central to human adaptability in unknown situations. In contrast, current autonomous agents often struggle in novel environments due to their limited capacity for adaptation. We hypothesize that metacognition is a critical missing ingredient in autonomous agents for the cognitive flexibility needed to tackle unfamiliar challenges. Given the broad scope of metacognitive abilities, we focus on competence awareness and strategy selection. To this end, we propose the Metacognition for Unknown Situations and Environments (MUSE) framework to integrate metacognitive processes of self-assessment and self-regulation into autonomous agents. We present two implementations of MUSE: one based on world modeling and another leveraging large language models (LLMs). Our system continually learns to assess its competence on a given task and uses this self-assessment to guide iterative cycles of strategy selection. MUSE agents demonstrate high competence awareness and significant improvements in self-regulation for solving novel, out-of-distribution tasks more effectively compared to model-based reinforcement learning and purely prompt-based LLM agent approaches. This work highlights the promise of approaches inspired by cognitive and neural systems in enabling autonomous agents to adapt to new environments while mitigating the heavy reliance on extensive training data and large models for the current models.