Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
This work addresses the problem of improving response selection in chatbots for developers and users, but it is incremental as it builds on existing pre-trained language models.
The paper tackles multi-turn response selection in retrieval-based chatbots by proposing Speaker-Aware BERT (SA-BERT) to incorporate speaker change information and a disentanglement strategy, achieving new state-of-the-art performances with large margins on five public datasets.
In this paper, we study the problem of employing pre-trained language models for multi-turn response selection in retrieval-based chatbots. A new model, named Speaker-Aware BERT (SA-BERT), is proposed in order to make the model aware of the speaker change information, which is an important and intrinsic property of multi-turn dialogues. Furthermore, a speaker-aware disentanglement strategy is proposed to tackle the entangled dialogues. This strategy selects a small number of most important utterances as the filtered context according to the speakers' information in them. Finally, domain adaptation is performed to incorporate the in-domain knowledge into pre-trained language models. Experiments on five public datasets show that our proposed model outperforms the present models on all metrics by large margins and achieves new state-of-the-art performances for multi-turn response selection.