Template-assisted Contrastive Learning of Task-oriented Dialogue Sentence Embeddings
This work addresses the challenge of annotating dialogue data for task-oriented systems, offering a more efficient approach by leveraging easier-to-obtain token-level annotations, though it is incremental in its application to existing contrastive learning frameworks.
The paper tackles the problem of learning high-quality sentence embeddings from dialogues with low annotation cost by introducing TaDSE, a method that uses template information and contrastive learning, achieving significant improvements over previous state-of-the-art methods on five benchmark datasets.
Learning high quality sentence embeddings from dialogues has drawn increasing attentions as it is essential to solve a variety of dialogue-oriented tasks with low annotation cost. Annotating and gathering utterance relationships in conversations are difficult, while token-level annotations, \eg, entities, slots and templates, are much easier to obtain. Other sentence embedding methods are usually sentence-level self-supervised frameworks and cannot utilize token-level extra knowledge. We introduce Template-aware Dialogue Sentence Embedding (TaDSE), a novel augmentation method that utilizes template information to learn utterance embeddings via self-supervised contrastive learning framework. We further enhance the effect with a synthetically augmented dataset that diversifies utterance-template association, in which slot-filling is a preliminary step. We evaluate TaDSE performance on five downstream benchmark dialogue datasets. The experiment results show that TaDSE achieves significant improvements over previous SOTA methods for dialogue. We further introduce a novel analytic instrument of semantic compression test, for which we discover a correlation with uniformity and alignment. Our code will be released upon acceptance.