Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Study
This work addresses the problem of evaluating and improving the general learning abilities of LLMs for researchers and developers, though it is incremental as it builds on existing cognitive and educational concepts.
The authors tackled the underexplored learning ability of large language models by introducing a cognitive framework that decomposes it into three dimensions—Learning from Instructor, Concept, and Experience—and conducted an empirical study, finding that interaction improves learning, conceptual understanding scales with model size, and LLMs are effective few-shot but not many-shot learners.
Large language models (LLMs) have shown impressive capabilities across tasks such as mathematics, coding, and reasoning, yet their learning ability, which is crucial for adapting to dynamic environments and acquiring new knowledge, remains underexplored. In this work, we address this gap by introducing a framework inspired by cognitive psychology and education. Specifically, we decompose general learning ability into three distinct, complementary dimensions: Learning from Instructor (acquiring knowledge via explicit guidance), Learning from Concept (internalizing abstract structures and generalizing to new contexts), and Learning from Experience (adapting through accumulated exploration and feedback). We conduct a comprehensive empirical study across the three learning dimensions and identify several insightful findings, such as (i) interaction improves learning; (ii) conceptual understanding is scale-emergent and benefits larger models; and (iii) LLMs are effective few-shot learners but not many-shot learners. Based on our framework and empirical findings, we introduce a benchmark that provides a unified and realistic evaluation of LLMs' general learning abilities across three learning cognition dimensions. It enables diagnostic insights and supports evaluation and development of more adaptive and human-like models.