Building Interpretable Interaction Trees for Deep NLP Models
This work addresses the interpretability challenge for researchers and practitioners using deep NLP models, though it appears incremental as it builds on existing Shapley value techniques.
The paper tackles the problem of understanding interactions between words in deep neural networks for NLP by proposing a method to construct interpretable interaction trees using Shapley values. Experimental results demonstrate the method's effectiveness in quantifying word interactions across BERT, ELMo, LSTM, CNN, and Transformer networks, providing new insights into these models.
This paper proposes a method to disentangle and quantify interactions among words that are encoded inside a DNN for natural language processing. We construct a tree to encode salient interactions extracted by the DNN. Six metrics are proposed to analyze properties of interactions between constituents in a sentence. The interaction is defined based on Shapley values of words, which are considered as an unbiased estimation of word contributions to the network prediction. Our method is used to quantify word interactions encoded inside the BERT, ELMo, LSTM, CNN, and Transformer networks. Experimental results have provided a new perspective to understand these DNNs, and have demonstrated the effectiveness of our method.