Latent Topology Induction for Understanding Contextualized Representations
This work provides insights into the internal mechanisms of language models, which is incremental for researchers in NLP and interpretability.
The authors tackled the problem of understanding the hidden topology of contextualized embeddings in large language models, showing that a latent network of states can be induced unsupervised to decompose the representation space into fine-grained linguistic properties and encode syntactic templates through state transitions.
In this work, we study the representation space of contextualized embeddings and gain insight into the hidden topology of large language models. We show there exists a network of latent states that summarize linguistic properties of contextualized representations. Instead of seeking alignments to existing well-defined annotations, we infer this latent network in a fully unsupervised way using a structured variational autoencoder. The induced states not only serve as anchors that mark the topology (neighbors and connectivity) of the representation manifold but also reveal the internal mechanism of encoding sentences. With the induced network, we: (1). decompose the representation space into a spectrum of latent states which encode fine-grained word meanings with lexical, morphological, syntactic and semantic information; (2). show state-state transitions encode rich phrase constructions and serve as the backbones of the latent space. Putting the two together, we show that sentences are represented as a traversal over the latent network where state-state transition chains encode syntactic templates and state-word emissions fill in the content. We demonstrate these insights with extensive experiments and visualizations.