Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking
This work addresses the challenge of data efficiency for researchers and practitioners in task-oriented dialogue systems, though it is incremental as it builds on existing in-context learning methods.
The paper tackles the problem of high data annotation costs in dialogue state tracking by proposing RefPyDST, which formulates DST as a Python programming task and uses diverse retrieval-augmented in-context learning, achieving state-of-the-art multi-domain joint-goal accuracy in zero and few-shot settings on MultiWOZ.
There has been significant interest in zero and few-shot learning for dialogue state tracking (DST) due to the high cost of collecting and annotating task-oriented dialogues. Recent work has demonstrated that in-context learning requires very little data and zero parameter updates, and even outperforms trained methods in the few-shot setting (Hu et al. 2022). We propose RefPyDST, which advances the state of the art with three advancements to in-context learning for DST. First, we formulate DST as a Python programming task, explicitly modeling language coreference as variable reference in Python. Second, since in-context learning depends highly on the context examples, we propose a method to retrieve a diverse set of relevant examples to improve performance. Finally, we introduce a novel re-weighting method during decoding that takes into account probabilities of competing surface forms, and produces a more accurate dialogue state prediction. We evaluate our approach using MultiWOZ and achieve state-of-the-art multi-domain joint-goal accuracy in zero and few-shot settings.