Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings
This addresses the problem for practitioners needing low-latency dialogue processing systems by enabling smaller models to match larger ones in accuracy, though it is incremental as it builds on existing preference learning methods.
The paper tackles the challenge of deploying large language models (LLMs) for real-time dialogue analysis by proposing a framework that combines LLM-generated labels with human annotations to train smaller, faster models, achieving accuracy improvements of over 2% in sentiment detection and over 1.5% in dialogue act classification.
Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. However, analyzing live dialogues in real-time necessitates low-latency processing systems, making it impractical to deploy models with billions of parameters due to latency constraints. As a result, practitioners often prefer smaller models with millions of parameters, trained on high-quality, human-annotated datasets. Yet, curating such datasets is both time-consuming and costly. Consequently, there is a growing need to combine the scalability of LLM-generated labels with the precision of human annotations, enabling fine-tuned smaller models to achieve both higher speed and accuracy comparable to larger models. In this paper, we introduce a simple yet effective framework to address this challenge. Our approach is specifically designed for per-utterance classification problems, which encompass tasks such as intent detection, dialogue state tracking, and more. To mitigate the impact of labeling errors from LLMs -- the primary source of inaccuracies in student models -- we propose a noise-reduced preference learning loss. Experimental results demonstrate that our method significantly improves accuracy across utterance-level dialogue tasks, including sentiment detection (over $2\%$), dialogue act classification (over $1.5\%$), etc.