Targeted Visualization of the Backbone of Encoder LLMs
This work addresses the need for explainable AI in NLP to detect risks like bias and adversarial attacks, but it is incremental as it adapts an existing method to a new domain.
The paper tackled the lack of global explainability methods for encoder LLMs like BERT by applying DeepView, a visualization technique from image classification, to NLP classifiers, demonstrating its use on BERT-based models with adversarial inputs and various training settings.
Attention based Large Language Models (LLMs) are the state-of-the-art in natural language processing (NLP). The two most common architectures are encoders such as BERT, and decoders like the GPT models. Despite the success of encoder models, on which we focus in this work, they also bear several risks, including issues with bias or their susceptibility for adversarial attacks, signifying the necessity for explainable AI to detect such issues. While there does exist various local explainability methods focusing on the prediction of single inputs, global methods based on dimensionality reduction for classification inspection, which have emerged in other domains and that go further than just using t-SNE in the embedding space, are not widely spread in NLP. To reduce this gap, we investigate the application of DeepView, a method for visualizing a part of the decision function together with a data set in two dimensions, to the NLP domain. While in previous work, DeepView has been used to inspect deep image classification models, we demonstrate how to apply it to BERT-based NLP classifiers and investigate its usability in this domain, including settings with adversarially perturbed input samples and pre-trained, fine-tuned, and multi-task models.