EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training
This work addresses the problem of data and model size constraints for Chinese dialogue systems, though it is incremental as it scales up existing methods.
The authors tackled the limitation of open-domain Chinese dialogue systems by proposing EVA, a system with a 2.8B-parameter pre-trained model, which outperforms other models in multi-turn interactions as shown by experiments.
Although pre-trained language models have remarkably enhanced the generation ability of dialogue systems, open-domain Chinese dialogue systems are still limited by the dialogue data and the model size compared with English ones. In this paper, we propose EVA, a Chinese dialogue system that contains the largest Chinese pre-trained dialogue model with 2.8B parameters. To build this model, we collect the largest Chinese dialogue dataset named WDC-Dialogue from various public social media. This dataset contains 1.4B context-response pairs and is used as the pre-training corpus of EVA. Extensive experiments on automatic and human evaluation show that EVA outperforms other Chinese pre-trained dialogue models especially in the multi-turn interaction of human-bot conversations.