Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
This addresses the interpretability issue for users of LSTMs in NLP, offering a method to understand model decisions without modifications, though it is incremental as it builds on existing interpretation techniques.
The paper tackles the problem of interpreting LSTMs as black boxes by introducing contextual decomposition (CD), an algorithm that extracts interactions between words to explain predictions, showing it reliably identifies contrasting sentiments and negations in sentiment analysis tasks on Yelp and SST datasets.
The driving force behind the recent success of LSTMs has been their ability to learn complex and non-linear relationships. Consequently, our inability to describe these relationships has led to LSTMs being characterized as black boxes. To this end, we introduce contextual decomposition (CD), an interpretation algorithm for analysing individual predictions made by standard LSTMs, without any changes to the underlying model. By decomposing the output of a LSTM, CD captures the contributions of combinations of words or variables to the final prediction of an LSTM. On the task of sentiment analysis with the Yelp and SST data sets, we show that CD is able to reliably identify words and phrases of contrasting sentiment, and how they are combined to yield the LSTM's final prediction. Using the phrase-level labels in SST, we also demonstrate that CD is able to successfully extract positive and negative negations from an LSTM, something which has not previously been done.