Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models
This research provides novel insights into the interpretation and potential lightweighting of LLMs for certain downstream tasks, which is significant for natural language processing researchers and practitioners.
This study tackled the problem of understanding the mechanisms and functional characteristics of large language models (LLMs) and found that they contain functional networks similar to those in the human brain, which are crucial for LLM performance, with masking key networks impairing performance by an unspecified amount. The study also found that retaining a subset of these networks can maintain effective operation.
In recent years, the rapid advancement of large language models (LLMs) in natural language processing has sparked significant interest among researchers to understand their mechanisms and functional characteristics. Although existing studies have attempted to explain LLM functionalities by identifying and interpreting specific neurons, these efforts mostly focus on individual neuron contributions, neglecting the fact that human brain functions are realized through intricate interaction networks. Inspired by cognitive neuroscience research on functional brain networks (FBNs), this study introduces a novel approach to investigate whether similar functional networks exist within LLMs. We use methods similar to those in the field of functional neuroimaging analysis to locate and identify functional networks in LLM. Experimental results show that, similar to the human brain, LLMs contain functional networks that frequently recur during operation. Further analysis shows that these functional networks are crucial for LLM performance. Masking key functional networks significantly impairs the model's performance, while retaining just a subset of these networks is adequate to maintain effective operation. This research provides novel insights into the interpretation of LLMs and the lightweighting of LLMs for certain downstream tasks. Code is available at https://github.com/WhatAboutMyStar/LLM_ACTIVATION.