Capability Self-Assessment: Teaching LLMs to Know Their Limits
For developers of reliable AI systems, this work addresses the critical problem of LLMs lacking self-awareness, enabling better delegation and decision-making.
Modern LLMs systematically overestimate their competence and attempt queries they cannot solve. Reinforcement learning teaches Capability Self-Assessment (CSA) effectively, significantly outperforming supervised fine-tuning while preserving original capabilities, and generalizes well out of distribution.
The ability to recognize one's own limitations and decide whether to solve a problem or delegate is fundamental for reliable intelligent systems. Yet we show that modern large language models systematically lack this ability: across diverse model families and scales, they overestimate their competence and attempt queries they cannot solve. We refer to this ability as Capability Self-Assessment (CSA) and formulate it as a policy-learning problem, aiming to improve self-assessment while preserving the model's original capabilities. Our results show that reinforcement learning teaches CSA effectively, significantly outperforming supervised fine-tuning while preserving original capabilities. In contrast, supervised fine-tuning severely degrades the capabilities the model is meant to assess. Moreover, learned self-assessment behavior generalizes well out of distribution, suggesting that CSA is a transferable model trait. Finally, CSA is practically useful: it improves local-cloud decision making at inference time and provides a signal for targeted data selection during training.