Using Captum to Explain Generative Language Models
This work addresses the need for interpretability tools for generative language models, but it is incremental as it builds on an existing library.
The paper tackles the problem of explaining generative language models by introducing new features in the Captum library, providing an overview of functionalities and example applications for understanding learned associations.
Captum is a comprehensive library for model explainability in PyTorch, offering a range of methods from the interpretability literature to enhance users' understanding of PyTorch models. In this paper, we introduce new features in Captum that are specifically designed to analyze the behavior of generative language models. We provide an overview of the available functionalities and example applications of their potential for understanding learned associations within generative language models.