Improving Context Modeling in Neural Topic Segmentation
This work addresses a key bottleneck in NLP tasks like topic segmentation, though it appears incremental as it builds on existing neural supervised approaches.
The authors tackled the problem of limited context modeling in neural topic segmentation by enhancing a hierarchical attention BiLSTM network with a coherence-related auxiliary task and restricted self-attention, resulting in a model that outperforms state-of-the-art approaches on three datasets and demonstrates robustness in domain transfer and multilingual scenarios.
Topic segmentation is critical in key NLP tasks and recent works favor highly effective neural supervised approaches. However, current neural solutions are arguably limited in how they model context. In this paper, we enhance a segmenter based on a hierarchical attention BiLSTM network to better model context, by adding a coherence-related auxiliary task and restricted self-attention. Our optimized segmenter outperforms SOTA approaches when trained and tested on three datasets. We also the robustness of our proposed model in domain transfer setting by training a model on a large-scale dataset and testing it on four challenging real-world benchmarks. Furthermore, we apply our proposed strategy to two other languages (German and Chinese), and show its effectiveness in multilingual scenarios.