Discursive Circuits: How Do Language Models Understand Discourse Relations?
This work addresses the problem of interpretability in language models for researchers in NLP and AI, but it is incremental as it builds on existing circuit discovery methods with a new task and dataset.
The paper tackled the problem of identifying which components in transformer language models are responsible for discourse understanding by hypothesizing sparse computational graphs called discursive circuits. The result showed that sparse circuits (approximately 0.2% of a full GPT-2 model) recovered discourse understanding in a specific task and generalized well to unseen discourse frameworks.
Which components in transformer language models are responsible for discourse understanding? We hypothesize that sparse computational graphs, termed as discursive circuits, control how models process discourse relations. Unlike simpler tasks, discourse relations involve longer spans and complex reasoning. To make circuit discovery feasible, we introduce a task called Completion under Discourse Relation (CuDR), where a model completes a discourse given a specified relation. To support this task, we construct a corpus of minimal contrastive pairs tailored for activation patching in circuit discovery. Experiments show that sparse circuits ($\approx 0.2\%$ of a full GPT-2 model) recover discourse understanding in the English PDTB-based CuDR task. These circuits generalize well to unseen discourse frameworks such as RST and SDRT. Further analysis shows lower layers capture linguistic features such as lexical semantics and coreference, while upper layers encode discourse-level abstractions. Feature utility is consistent across frameworks (e.g., coreference supports Expansion-like relations).