Visualizing Attention in Transformer-Based Language Representation Models
This tool helps researchers and practitioners interpret complex language models, but it is incremental as it builds on existing visualization work.
The authors developed an open-source tool for visualizing multi-head self-attention in Transformer-based models like BERT and GPT-2, enabling analysis at three granularity levels to aid in model interpretation.
We present an open-source tool for visualizing multi-head self-attention in Transformer-based language representation models. The tool extends earlier work by visualizing attention at three levels of granularity: the attention-head level, the model level, and the neuron level. We describe how each of these views can help to interpret the model, and we demonstrate the tool on the BERT model and the OpenAI GPT-2 model. We also present three use cases for analyzing GPT-2: detecting model bias, identifying recurring patterns, and linking neurons to model behavior.