Context-Aware Language Modeling for Goal-Oriented Dialogue Systems
This work addresses the challenge of steering language models for task completion without sacrificing quality, which is incremental as it builds on existing control techniques.
The paper tackled the problem of balancing fluent language generation with task-specific control in goal-oriented dialogue systems, achieving a 7% improvement in task success over the state-of-the-art on a flight-booking task.
Goal-oriented dialogue systems face a trade-off between fluent language generation and task-specific control. While supervised learning with large language models is capable of producing realistic text, how to steer such responses towards completing a specific task without sacrificing language quality remains an open question. In this work, we formulate goal-oriented dialogue as a partially observed Markov decision process, interpreting the language model as a representation of both the dynamics and the policy. This view allows us to extend techniques from learning-based control, such as task relabeling, to derive a simple and effective method to finetune language models in a goal-aware way, leading to significantly improved task performance. We additionally introduce a number of training strategies that serve to better focus the model on the task at hand. We evaluate our method, Context-Aware Language Models (CALM), on a practical flight-booking task using AirDialogue. Empirically, CALM outperforms the state-of-the-art method by 7% in terms of task success, matching human-level task performance.