DD-GEPA: Prompt Optimization for Dialogue Disentanglement Focusing on Task Instruction and Utterance Representation
For researchers working on dialogue systems, this work offers an automatic prompt optimization method that improves LLM performance on a challenging task, though the gains are incremental.
The paper addresses dialogue disentanglement in multi-party chats, where LLMs currently achieve low accuracy. By decomposing and optimizing prompts using the GEPA method, they improve accuracy over original and hand-crafted prompts on benchmark datasets.
Multi-party chat often contains interleaved dialogues because multiple participants can discuss different topics at the same time. Dialogue disentanglement addresses this problem by separating an entangled utterance sequence into coherent dialogues. While large language models (LLMs) are promising for this task, they still struggle with dialogue disentanglement and achieve low accuracy. This paper proposes an automatic prompt optimization for LLM based dialogue disentanglement. We decompose the prompt into three components: task instruction, utterance representation, and output instruction, and optimize them using GEPA, an optimization method for compound AI systems. Experiments on benchmark datasets show that the optimized prompts improve dialogue disentanglement accuracy over the original prompts and can surpass hand crafted prompts.