Clustering in pure-attention hardmax transformers and its role in sentiment analysis
This work addresses the challenge of interpretability in transformers for researchers and practitioners in natural language processing, though it is incremental as it builds on existing transformer frameworks with a focus on theoretical analysis.
The authors tackled the problem of understanding the mathematical behavior of transformers with hardmax self-attention by analyzing them as dynamical systems, showing that inputs converge to clustered equilibria determined by leader points. They applied this theoretical insight to sentiment analysis, using an interpretable transformer model that clusters meaningless words around meaningful leader words.
Transformers are extremely successful machine learning models whose mathematical properties remain poorly understood. Here, we rigorously characterize the behavior of transformers with hardmax self-attention and normalization sublayers as the number of layers tends to infinity. By viewing such transformers as discrete-time dynamical systems describing the evolution of points in a Euclidean space, and thanks to a geometric interpretation of the self-attention mechanism based on hyperplane separation, we show that the transformer inputs asymptotically converge to a clustered equilibrium determined by special points called leaders. We then leverage this theoretical understanding to solve sentiment analysis problems from language processing using a fully interpretable transformer model, which effectively captures `context' by clustering meaningless words around leader words carrying the most meaning. Finally, we outline remaining challenges to bridge the gap between the mathematical analysis of transformers and their real-life implementation.