Interactively Providing Explanations for Transformer Language Models
This addresses the need for interpretability in NLP for researchers and practitioners, offering a more interactive and human-in-the-loop approach, though it is incremental as it builds on existing prototype methods.
The paper tackles the problem of opacity in transformer language models by proposing an architecture that integrates prototype networks to explain the reasoning process behind decisions, achieving performance comparable to several language models and enabling learning from user interactions.
Transformer language models are state of the art in a multitude of NLP tasks. Despite these successes, their opaqueness remains problematic. Recent methods aiming to provide interpretability and explainability to black-box models primarily focus on post-hoc explanations of (sometimes spurious) input-output correlations. Instead, we emphasize using prototype networks directly incorporated into the model architecture and hence explain the reasoning process behind the network's decisions. Our architecture performs on par with several language models and, moreover, enables learning from user interactions. This not only offers a better understanding of language models but uses human capabilities to incorporate knowledge outside of the rigid range of purely data-driven approaches.