PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism
This addresses the issue of low diversity and relevance in generative chatbots, but it is incremental as it builds on existing RNN-based methods.
The authors tackled the problem of generating diverse and relevant responses in multi-turn dialogue chatbots by proposing a Pseudo-Variational Gated Recurrent Unit (PVGRU) component, which improved diversity and relevance on two benchmark datasets.
We investigate response generation for multi-turn dialogue in generative-based chatbots. Existing generative models based on RNNs (Recurrent Neural Networks) usually employ the last hidden state to summarize the sequences, which makes models unable to capture the subtle variability observed in different dialogues and cannot distinguish the differences between dialogues that are similar in composition. In this paper, we propose a Pseudo-Variational Gated Recurrent Unit (PVGRU) component without posterior knowledge through introducing a recurrent summarizing variable into the GRU, which can aggregate the accumulated distribution variations of subsequences. PVGRU can perceive the subtle semantic variability through summarizing variables that are optimized by the devised distribution consistency and reconstruction objectives. In addition, we build a Pseudo-Variational Hierarchical Dialogue (PVHD) model based on PVGRU. Experimental results demonstrate that PVGRU can broadly improve the diversity and relevance of responses on two benchmark datasets.