The Mechanics of Conceptual Interpretation in GPT Models: Interpretative Insights
This work addresses the challenge of interpreting and editing knowledge in LLMs for improved accuracy and safety, but it is incremental as it builds on existing knowledge editing techniques.
The paper tackled the problem of understanding how large language models process semantic information by analyzing components like MLP and MHA layers, revealing distinct patterns such as key-value retrieval in MLPs and distributed semantic integration in MHAs.
Locating and editing knowledge in large language models (LLMs) is crucial for enhancing their accuracy, safety, and inference rationale. We introduce ``concept editing'', an innovative variation of knowledge editing that uncovers conceptualisation mechanisms within these models. Using the reverse dictionary task, inference tracing, and input abstraction, we analyse the Multi-Layer Perceptron (MLP), Multi-Head Attention (MHA), and hidden state components of transformer models. Our results reveal distinct patterns: MLP layers employ key-value retrieval mechanism and context-dependent processing, which are highly associated with relative input tokens. MHA layers demonstrate a distributed nature with significant higher-level activations, suggesting sophisticated semantic integration. Hidden states emphasise the importance of the last token and top layers in the inference process. We observe evidence of gradual information building and distributed representation. These observations elucidate how transformer models process semantic information, paving the way for targeted interventions and improved interpretability techniques. Our work highlights the complex, layered nature of semantic processing in LLMs and the challenges of isolating and modifying specific concepts within these models.