A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training
This work is significant for improving the accuracy of retrieval-based chatbots by enhancing their reasoning ability in multi-turn conversations, benefiting users seeking more coherent and relevant responses.
This paper addresses multi-turn response selection in retrieval-based chatbots by proposing a graph-reasoning network (GRN). The GRN, after customized pre-training with ALBERT, significantly outperforms strong baselines on two conversational reasoning datasets, achieving near human-level performance.
We investigate response selection for multi-turn conversation in retrieval-based chatbots. Existing studies pay more attention to the matching between utterances and responses by calculating the matching score based on learned features, leading to insufficient model reasoning ability. In this paper, we propose a graph-reasoning network (GRN) to address the problem. GRN first conducts pre-training based on ALBERT using next utterance prediction and utterance order prediction tasks specifically devised for response selection. These two customized pre-training tasks can endow our model with the ability of capturing semantical and chronological dependency between utterances. We then fine-tune the model on an integrated network with sequence reasoning and graph reasoning structures. The sequence reasoning module conducts inference based on the highly summarized context vector of utterance-response pairs from the global perspective. The graph reasoning module conducts the reasoning on the utterance-level graph neural network from the local perspective. Experiments on two conversational reasoning datasets show that our model can dramatically outperform the strong baseline methods and can achieve performance which is close to human-level.