Understanding Language Model Circuits through Knowledge Editing
This work provides insights into knowledge representation in language models, aiding interpretability and safety research, though it is incremental as it builds on existing circuit identification methods.
The researchers tackled the problem of understanding how knowledge is structured within critical subnetworks (circuits) of language models by conducting systematic knowledge editing experiments on GPT-2, revealing patterns in circuit responses, knowledge distribution, and architectural composition.
Recent advances in language model interpretability have identified circuits, critical subnetworks that replicate model behaviors, yet how knowledge is structured within these crucial subnetworks remains opaque. To gain an understanding toward the knowledge in the circuits, we conduct systematic knowledge editing experiments on the circuits of the GPT-2 language model. Our analysis reveals intriguing patterns in how circuits respond to editing attempts, the extent of knowledge distribution across network components, and the architectural composition of knowledge-bearing circuits. These findings offer insights into the complex relationship between model circuits and knowledge representation, deepening the understanding of how information is organized within language models. Our findings offer novel insights into the ``meanings'' of the circuits, and introduce directions for further interpretability and safety research of language models.