From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions
This work addresses the interpretability of Transformer models in NLP, providing insights into how they encode syntax, though it is incremental as it builds on existing analysis of attention mechanisms.
The authors investigated whether Transformer NMT encoders capture syntactic structure by analyzing self-attention patterns across three source languages, finding that many heads exhibit phrase-like sequences resembling syntax. They developed a method to quantify syntactic information by building phrase-structure trees from these sequences and evaluated them against treebanks, reporting precision and recall metrics.
We inspect the multi-head self-attention in Transformer NMT encoders for three source languages, looking for patterns that could have a syntactic interpretation. In many of the attention heads, we frequently find sequences of consecutive states attending to the same position, which resemble syntactic phrases. We propose a transparent deterministic method of quantifying the amount of syntactic information present in the self-attentions, based on automatically building and evaluating phrase-structure trees from the phrase-like sequences. We compare the resulting trees to existing constituency treebanks, both manually and by computing precision and recall.