LSTM-based Mixture-of-Experts for Knowledge-Aware Dialogues
This is an incremental improvement for dialogue systems, aiming to enhance performance by blending existing models without specifying concrete gains.
The paper tackles the problem of integrating multiple language models for dialogue by introducing an LSTM-based mixture-of-experts method that combines a neural chat model and a neural question-answering model to leverage their respective strengths in conversation and knowledge retrieval.
We introduce an LSTM-based method for dynamically integrating several word-prediction experts to obtain a conditional language model which can be good simultaneously at several subtasks. We illustrate this general approach with an application to dialogue where we integrate a neural chat model, good at conversational aspects, with a neural question-answering model, good at retrieving precise information from a knowledge-base, and show how the integration combines the strengths of the independent components. We hope that this focused contribution will attract attention on the benefits of using such mixtures of experts in NLP.