Iterative Recursive Attention Model for Interpretable Sequence Classification
This work addresses the problem of interpretability in sequence classification for NLP researchers and practitioners, offering an incremental improvement over existing attention mechanisms.
The paper tackles the limited interpretability of standard attention models for tasks requiring multiple inference steps by introducing an iterative recursive attention model that builds incremental representations through reused queries. The model achieves performance close to state-of-the-art on sentiment classification datasets while providing easily interpretable aspect identification and combination.
Natural language processing has greatly benefited from the introduction of the attention mechanism. However, standard attention models are of limited interpretability for tasks that involve a series of inference steps. We describe an iterative recursive attention model, which constructs incremental representations of input data through reusing results of previously computed queries. We train our model on sentiment classification datasets and demonstrate its capacity to identify and combine different aspects of the input in an easily interpretable manner, while obtaining performance close to the state of the art.