Latent Concept-based Explanation of NLP Models
This work addresses the problem of model interpretability for NLP researchers and practitioners, offering a novel approach that is incremental in advancing explanation methods.
The paper tackles the challenge of interpreting deep learning models in NLP by introducing the Latent Concept Attribution method (LACOAT), which generates explanations based on latent concepts to address the limitations of word-based explanations, resulting in improved interpretability as demonstrated through experiments on benchmark datasets.
Interpreting and understanding the predictions made by deep learning models poses a formidable challenge due to their inherently opaque nature. Many previous efforts aimed at explaining these predictions rely on input features, specifically, the words within NLP models. However, such explanations are often less informative due to the discrete nature of these words and their lack of contextual verbosity. To address this limitation, we introduce the Latent Concept Attribution method (LACOAT), which generates explanations for predictions based on latent concepts. Our foundational intuition is that a word can exhibit multiple facets, contingent upon the context in which it is used. Therefore, given a word in context, the latent space derived from our training process reflects a specific facet of that word. LACOAT functions by mapping the representations of salient input words into the training latent space, allowing it to provide latent context-based explanations of the prediction.