Child vs. machine language learning: Can the logical structure of human language unleash LLMs?
This addresses the problem of improving LLM performance by highlighting structural differences between human and machine learning, though it is incremental as it builds on existing critiques.
The paper argues that human language learning differs from current LLM training, leading to different biases, and provides evidence from German plural formation showing that even powerful LLMs miss logical aspects of language that humans handle easily.
We argue that human language learning proceeds in a manner that is different in nature from current approaches to training LLMs, predicting a difference in learning biases. We then present evidence from German plural formation by LLMs that confirm our hypothesis that even very powerful implementations produce results that miss aspects of the logic inherent to language that humans have no problem with. We conclude that attention to the different structures of human language and artificial neural networks is likely to be an avenue to improve LLM performance.