On the Analogy between Human Brain and LLMs: Spotting Key Neurons in Grammar Perception
This provides insights into the internal mechanisms of LLMs for researchers in AI interpretability and computational linguistics, though it is incremental as it extends known neuroscience analogies to a specific model.
The paper tackled the problem of understanding how Large Language Models process grammatical categories by identifying key neurons in Llama 3 associated with part-of-speech tag prediction, and showed that activation patterns of these neurons can reliably predict part-of-speech tags on fresh data with a classifier achieving high accuracy.
Artificial Neural Networks, the building blocks of AI, were inspired by the human brain's network of neurons. Over the years, these networks have evolved to replicate the complex capabilities of the brain, allowing them to handle tasks such as image and language processing. In the realm of Large Language Models, there has been a keen interest in making the language learning process more akin to that of humans. While neuroscientific research has shown that different grammatical categories are processed by different neurons in the brain, we show that LLMs operate in a similar way. Utilizing Llama 3, we identify the most important neurons associated with the prediction of words belonging to different part-of-speech tags. Using the achieved knowledge, we train a classifier on a dataset, which shows that the activation patterns of these key neurons can reliably predict part-of-speech tags on fresh data. The results suggest the presence of a subspace in LLMs focused on capturing part-of-speech tag concepts, resembling patterns observed in lesion studies of the brain in neuroscience.