Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations
This addresses a less explored but important problem for open-domain conversational AI, though it is incremental as it builds on existing distillation and multi-tower architectures.
The paper tackles coarse-grained response selection in retrieval-based dialogue systems by proposing a Contextual Fine-to-Coarse distilled model, which achieves significant improvements over baseline methods on two new datasets based on Reddit and Twitter data.
We study the problem of coarse-grained response selection in retrieval-based dialogue systems. The problem is equally important with fine-grained response selection, but is less explored in existing literature. In this paper, we propose a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response selection in open-domain conversations. In our CFC model, dense representations of query, candidate response and corresponding context is learned based on the multi-tower architecture, and more expressive knowledge learned from the one-tower architecture (fine-grained) is distilled into the multi-tower architecture (coarse-grained) to enhance the performance of the retriever. To evaluate the performance of our proposed model, we construct two new datasets based on the Reddit comments dump and Twitter corpus. Extensive experimental results on the two datasets show that the proposed methods achieve a significant improvement over all evaluation metrics compared with traditional baseline methods.