No Such Thing as a General Learner: Language models and their dual optimization
This work addresses theoretical debates in cognitive science and AI about the role of LLMs in understanding human language acquisition, but it is incremental as it builds on existing critiques of general learning.
The paper argues that neither humans nor large language models (LLMs) are general learners, proposing a dual-optimization process for LLMs involving training and selection akin to natural selection, and concludes that LLM performance does not easily inform debates on human cognitive biases in language acquisition.
What role can the otherwise successful Large Language Models (LLMs) play in the understanding of human cognition, and in particular in terms of informing language acquisition debates? To contribute to this question, we first argue that neither humans nor LLMs are general learners, in a variety of senses. We make a novel case for how in particular LLMs follow a dual-optimization process: they are optimized during their training (which is typically compared to language acquisition), and modern LLMs have also been selected, through a process akin to natural selection in a species. From this perspective, we argue that the performance of LLMs, whether similar or dissimilar to that of humans, does not weigh easily on important debates about the importance of human cognitive biases for language.