PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning
This addresses the problem of creating more effective and diverse chatbots for general users, though it appears incremental as it builds on existing curriculum learning and latent variable approaches.
The authors tackled building a high-quality open-domain chatbot by introducing PLATO-2, which uses a two-stage curriculum learning process involving coarse-grained generation and fine-grained models with latent variables, achieving new state-of-the-art results in evaluations on Chinese and English data.
To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning. There are two stages involved in the learning process. In the first stage, a coarse-grained generation model is trained to learn response generation under the simplified framework of one-to-one mapping. In the second stage, a fine-grained generative model augmented with latent variables and an evaluation model are further trained to generate diverse responses and to select the best response, respectively. PLATO-2 was trained on both Chinese and English data, whose effectiveness and superiority are verified through comprehensive evaluations, achieving new state-of-the-art results.