Visual Analytics for Generative Transformer Models
This work addresses interpretability issues for researchers and practitioners using transformer-based generative models, though it is incremental as it extends visualization methods from encoder-based to encoder-decoder and decoder-only models.
The paper tackles the challenge of interpreting black-box transformer-based generative models by introducing a visual analytical framework, and demonstrates its feasibility through three case studies on real-world NLP problems.
While transformer-based models have achieved state-of-the-art results in a variety of classification and generation tasks, their black-box nature makes them challenging for interpretability. In this work, we present a novel visual analytical framework to support the analysis of transformer-based generative networks. In contrast to previous work, which has mainly focused on encoder-based models, our framework is one of the first dedicated to supporting the analysis of transformer-based encoder-decoder models and decoder-only models for generative and classification tasks. Hence, we offer an intuitive overview that allows the user to explore different facets of the model through interactive visualization. To demonstrate the feasibility and usefulness of our framework, we present three detailed case studies based on real-world NLP research problems.