Lexical Knowledge Internalization for Neural Dialog Generation
This work addresses the challenge of knowledge grounding in dialog generation for AI systems, presenting an incremental improvement over existing knowledge-grounded dialog methods.
The paper tackles the problem of integrating lexical knowledge into neural dialog models by proposing knowledge internalization (KI), which internally incorporates token-level knowledge into model parameters, and demonstrates its effectiveness across various datasets and model structures.
We propose knowledge internalization (KI), which aims to complement the lexical knowledge into neural dialog models. Instead of further conditioning the knowledge-grounded dialog (KGD) models on externally retrieved knowledge, we seek to integrate knowledge about each input token internally into the model's parameters. To tackle the challenge due to the large scale of lexical knowledge, we adopt the contrastive learning approach and create an effective token-level lexical knowledge retriever that requires only weak supervision mined from Wikipedia. We demonstrate the effectiveness and general applicability of our approach on various datasets and diversified model structures.