Dialogue Language Model with Large-Scale Persona Data Engineering
This work addresses persona consistency in open-domain dialogue systems, which is crucial for applications like ChatGPT, but it is incremental as it builds on existing large-scale pre-training methods.
The study tackled the problem of limited scale and diversity in persona dialogue datasets by introducing PPDS, an open-domain persona dialogue system that uses extensive generative pre-training and a persona extraction model to generate large datasets, resulting in superior response quality and persona consistency as shown in evaluations.
Maintaining persona consistency is paramount in the application of open-domain dialogue systems, as exemplified by models like ChatGPT. Despite significant advancements, the limited scale and diversity of current persona dialogue datasets remain challenges to achieving robust persona-consistent dialogue models. In this study, drawing inspiration from the success of large-scale pre-training, we introduce PPDS, an open-domain persona dialogue system that employs extensive generative pre-training on a persona dialogue dataset to enhance persona consistency. Specifically, we present a persona extraction model designed to autonomously and precisely generate vast persona dialogue datasets. Additionally, we unveil a pioneering persona augmentation technique to address the invalid persona bias inherent in the constructed dataset. Both quantitative and human evaluations consistently highlight the superior response quality and persona consistency of our proposed model, underscoring its effectiveness.