NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation
This addresses the issue of generating more engaging and context-aware responses in dialogue systems, though it appears incremental as it builds on existing seq2seq frameworks.
The paper tackled the problem of seq2seq dialogue models favoring short generic responses by proposing a method to generate responses that smoothly connect both preceding and following dialogue through mutual information maximization, validated on two datasets with improved contextual relevance and interactivity.
Sequence-to-Sequence (seq2seq) models have become overwhelmingly popular in building end-to-end trainable dialogue systems. Though highly efficient in learning the backbone of human-computer communications, they suffer from the problem of strongly favoring short generic responses. In this paper, we argue that a good response should smoothly connect both the preceding dialogue history and the following conversations. We strengthen this connection through mutual information maximization. To sidestep the non-differentiability of discrete natural language tokens, we introduce an auxiliary continuous code space and map such code space to a learnable prior distribution for generation purpose. Experiments on two dialogue datasets validate the effectiveness of our model, where the generated responses are closely related to the dialogue context and lead to more interactive conversations.