Federated Natural Language Generation for Personalized Dialogue System
This work addresses the dilemma of model adaptation and data privacy in persona-based dialogue systems for users needing personalized interactions, representing an incremental improvement by applying federated learning to a known bottleneck.
The paper tackles the problem of inconsistency and lack of coherent personality in neural conversational models by proposing a Federated Natural Language Generation (FedNLG) framework, which learns personalized representations from distributed datasets to implement personalized dialogue systems efficiently and safely, achieving an accuracy-privacy balance.
Neural conversational models have long suffered from the problem of inconsistency and lacking coherent personality. To address the issue, persona-based models capturing individual characteristics have been proposed, but they still face the dilemma of model adaption and data privacy. To break this dilemma, we propose a novel Federated Natural Language Generation (FedNLG) framework, which learns personalized representations from various dataset on distributed devices, and thus implements the personalized dialogue system efficiently and safely. FedNLG first pre-trains parameters of standard neural conversational model over a large dialogue corpus, and then fine-tune the model parameters and persona embeddings on specific datasets, in a federated manner. Thus, the model could simultaneously learn the persona embeddings in local clients and learn shared model parameters by federated aggregation, which achieves accuracyprivacy balance. By conducting extensive experiments, we demonstrate the effectiveness of our model by pre-training model over Cornell Movie-Dialogs Corpus and fine-tuning the model over two TV series dataset.