Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention
This work addresses a key problem in NLP for improving text understanding, but it is incremental as it builds on existing attention mechanisms and memory methods.
The paper tackles the challenge of recognizing implicit discourse relations in natural language processing by proposing neural networks with multi-level attention (NNMA) that mimic repeated reading to dynamically focus on key words, achieving state-of-the-art results on the PDTB dataset.
Recognizing implicit discourse relations is a challenging but important task in the field of Natural Language Processing. For such a complex text processing task, different from previous studies, we argue that it is necessary to repeatedly read the arguments and dynamically exploit the efficient features useful for recognizing discourse relations. To mimic the repeated reading strategy, we propose the neural networks with multi-level attention (NNMA), combining the attention mechanism and external memories to gradually fix the attention on some specific words helpful to judging the discourse relations. Experiments on the PDTB dataset show that our proposed method achieves the state-of-art results. The visualization of the attention weights also illustrates the progress that our model observes the arguments on each level and progressively locates the important words.