Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues?
This work addresses the dependency on external knowledge sources for knowledge-grounded dialogue generation, potentially simplifying the architecture for researchers and practitioners in natural language processing.
This paper investigates whether knowledge stored in pre-trained language models (PLMs) is sufficient for grounding open-domain dialogues without external knowledge sources. They found that fine-tuning PLMs with a small number of knowledge-infused dialogues allowed them to surpass state-of-the-art models reliant on external knowledge in both automatic and human evaluations.
We study knowledge-grounded dialogue generation with pre-trained language models. Instead of pursuing new state-of-the-art on benchmarks, we try to understand if the knowledge stored in parameters of the pre-trained models is already enough to ground open domain dialogues, and thus allows us to get rid of the dependency on external knowledge sources in generation. Through extensive experiments on benchmarks, we find that by fine-tuning with a few dialogues containing knowledge, the pre-trained language models can outperform the state-of-the-art model that requires external knowledge in automatic evaluation and human judgment, suggesting a positive answer to the question we raised.