MASE: Interpretable NLP Models via Model-Agnostic Saliency Estimation
This addresses interpretability for NLP practitioners, but it is incremental as it builds on existing saliency methods with a novel perturbation approach.
The paper tackled the problem of interpreting deep neural networks in NLP by introducing the Model-gnostic Saliency Estimation (MASE) framework, which uses Normalized Linear Gaussian Perturbations on embeddings to provide local explanations without requiring model architecture details, and results show it outperforms other methods in Delta Accuracy.
Deep neural networks (DNNs) have made significant strides in Natural Language Processing (NLP), yet their interpretability remains elusive, particularly when evaluating their intricate decision-making processes. Traditional methods often rely on post-hoc interpretations, such as saliency maps or feature visualization, which might not be directly applicable to the discrete nature of word data in NLP. Addressing this, we introduce the Model-agnostic Saliency Estimation (MASE) framework. MASE offers local explanations for text-based predictive models without necessitating in-depth knowledge of a model's internal architecture. By leveraging Normalized Linear Gaussian Perturbations (NLGP) on the embedding layer instead of raw word inputs, MASE efficiently estimates input saliency. Our results indicate MASE's superiority over other model-agnostic interpretation methods, especially in terms of Delta Accuracy, positioning it as a promising tool for elucidating the operations of text-based models in NLP.