Def-DTS: Deductive Reasoning for Open-domain Dialogue Topic Segmentation
This addresses data shortage and labeling ambiguity in DTS for NLP downstream tasks, representing an incremental improvement through the application of LLM reasoning techniques.
The paper tackled the problem of Dialogue Topic Segmentation (DTS) by introducing Def-DTS, which uses LLM-based deductive reasoning to improve segmentation performance, resulting in consistent outperformance over traditional and state-of-the-art methods with a focus on reducing type 2 error.
Dialogue Topic Segmentation (DTS) aims to divide dialogues into coherent segments. DTS plays a crucial role in various NLP downstream tasks, but suffers from chronic problems: data shortage, labeling ambiguity, and incremental complexity of recently proposed solutions. On the other hand, Despite advances in Large Language Models (LLMs) and reasoning strategies, these have rarely been applied to DTS. This paper introduces Def-DTS: Deductive Reasoning for Open-domain Dialogue Topic Segmentation, which utilizes LLM-based multi-step deductive reasoning to enhance DTS performance and enable case study using intermediate result. Our method employs a structured prompting approach for bidirectional context summarization, utterance intent classification, and deductive topic shift detection. In the intent classification process, we propose the generalizable intent list for domain-agnostic dialogue intent classification. Experiments in various dialogue settings demonstrate that Def-DTS consistently outperforms traditional and state-of-the-art approaches, with each subtask contributing to improved performance, particularly in reducing type 2 error. We also explore the potential for autolabeling, emphasizing the importance of LLM reasoning techniques in DTS.