Lost in Translation: The Algorithmic Gap Between LMs and the Brain
This work addresses the gap between artificial language models and human cognition for researchers in AI and neuroscience, but it is incremental as it reviews existing ideas without presenting new empirical results.
This paper tackles the problem of understanding the differences between language models and human brain processing, examining gaps and overlaps to inform the development of more biologically plausible models. It explores insights from neuroscience and scaling laws to bridge this gap, aiming to advance both AI and cognitive understanding.
Language Models (LMs) have achieved impressive performance on various linguistic tasks, but their relationship to human language processing in the brain remains unclear. This paper examines the gaps and overlaps between LMs and the brain at different levels of analysis, emphasizing the importance of looking beyond input-output behavior to examine and compare the internal processes of these systems. We discuss how insights from neuroscience, such as sparsity, modularity, internal states, and interactive learning, can inform the development of more biologically plausible language models. Furthermore, we explore the role of scaling laws in bridging the gap between LMs and human cognition, highlighting the need for efficiency constraints analogous to those in biological systems. By developing LMs that more closely mimic brain function, we aim to advance both artificial intelligence and our understanding of human cognition.