A Hierarchical Decoder with Three-level Hierarchical Attention to Generate Abstractive Summaries of Interleaved Texts
This work addresses the challenge of summarizing interleaved texts for users needing quick overviews, offering a significant improvement over existing methods.
The paper tackles the problem of generating abstractive summaries from interleaved texts, such as online chats, by proposing an end-to-end hierarchical encoder-decoder system with a novel three-level attention mechanism, which outperforms a state-of-the-art two-step system by 20-40% on multiple datasets.
Interleaved texts, where posts belonging to different threads occur in one sequence, are a common occurrence, e.g., online chat conversations. To quickly obtain an overview of such texts, existing systems first disentangle the posts by threads and then extract summaries from those threads. The major issues with such systems are error propagation and non-fluent summary. To address those, we propose an end-to-end trainable hierarchical encoder-decoder system. We also introduce a novel hierarchical attention mechanism which combines three levels of information from an interleaved text, i.e, posts, phrases and words, and implicitly disentangles the threads. We evaluated the proposed system on multiple interleaved text datasets, and it out-performs a SOTA two-step system by 20-40%.