DialAug: Mixing up Dialogue Contexts in Contrastive Learning for Robust Conversational Modeling
This work addresses robustness issues in conversational AI for retrieval-based systems, offering an incremental improvement through enhanced data augmentation and contrastive learning.
The paper tackles the problem of limited generalization in retrieval-based conversational systems by proposing DialAug, a framework that uses contrastive learning with augmented dialogue contexts to learn robust representations. The result is significant improvements over baseline BERT-based ranking architectures on four benchmark datasets, with the novel ConMix augmentation method outperforming previous approaches.
Retrieval-based conversational systems learn to rank response candidates for a given dialogue context by computing the similarity between their vector representations. However, training on a single textual form of the multi-turn context limits the ability of a model to learn representations that generalize to natural perturbations seen during inference. In this paper we propose a framework that incorporates augmented versions of a dialogue context into the learning objective. We utilize contrastive learning as an auxiliary objective to learn robust dialogue context representations that are invariant to perturbations injected through the augmentation method. We experiment with four benchmark dialogue datasets and demonstrate that our framework combines well with existing augmentation methods and can significantly improve over baseline BERT-based ranking architectures. Furthermore, we propose a novel data augmentation method, ConMix, that adds token level perturbations through stochastic mixing of tokens from other contexts in the batch. We show that our proposed augmentation method outperforms previous data augmentation approaches, and provides dialogue representations that are more robust to common perturbations seen during inference.