Considering Likelihood in NLP Classification Explanations with Occlusion and Language Modeling
This addresses the need for more reliable explanations in NLP models, though it is incremental as it builds on existing occlusion methods.
The paper tackles the problem that occlusion-based explanation methods in NLP often produce invalid or syntactically incorrect language data, and proposes OLM, a novel method combining occlusion with language models to sample valid replacements with high likelihood, showing improved explanation quality.
Recently, state-of-the-art NLP models gained an increasing syntactic and semantic understanding of language, and explanation methods are crucial to understand their decisions. Occlusion is a well established method that provides explanations on discrete language data, e.g. by removing a language unit from an input and measuring the impact on a model's decision. We argue that current occlusion-based methods often produce invalid or syntactically incorrect language data, neglecting the improved abilities of recent NLP models. Furthermore, gradient-based explanation methods disregard the discrete distribution of data in NLP. Thus, we propose OLM: a novel explanation method that combines occlusion and language models to sample valid and syntactically correct replacements with high likelihood, given the context of the original input. We lay out a theoretical foundation that alleviates these weaknesses of other explanation methods in NLP and provide results that underline the importance of considering data likelihood in occlusion-based explanation.