MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems
This addresses the problem of over-reliance on annotated data and complex designs for developers of task-oriented dialogue systems, offering a more efficient and data-efficient solution.
The paper tackles the complexity and data dependency in task-oriented dialogue systems by proposing Minimalist Transfer Learning (MinTL), which simplifies design and uses Levenshtein belief spans for efficient state tracking, achieving new state-of-the-art results in end-to-end response generation and robust performance with only 20% training data.
In this paper, we propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems and alleviate the over-dependency on annotated data. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models, and jointly learn dialogue state tracking and dialogue response generation. Unlike previous approaches, which use a copy mechanism to "carryover" the old dialogue states to the new one, we introduce Levenshtein belief spans (Lev), that allows efficient dialogue state tracking with a minimal generation length. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ. Extensive experiments demonstrate that: 1) our systems establish new state-of-the-art results on end-to-end response generation, 2) MinTL-based systems are more robust than baseline methods in the low resource setting, and they achieve competitive results with only 20\% training data, and 3) Lev greatly improves the inference efficiency.