Variational Memory Encoder-Decoder
This addresses the challenge of generating diverse and coherent responses in conversational AI, which is an incremental improvement over existing neural encoder-decoder models.
The paper tackled the problem of generating variable yet coherent conversational responses by proposing the Variational Memory Encoder-Decoder (VMED), which uses external memory as a mixture model to inject variability, and it achieved significant improvements over other approaches in metric-based and qualitative evaluations.
Introducing variability while maintaining coherence is a core task in learning to generate utterances in conversation. Standard neural encoder-decoder models and their extensions using conditional variational autoencoder often result in either trivial or digressive responses. To overcome this, we explore a novel approach that injects variability into neural encoder-decoder via the use of external memory as a mixture model, namely Variational Memory Encoder-Decoder (VMED). By associating each memory read with a mode in the latent mixture distribution at each timestep, our model can capture the variability observed in sequential data such as natural conversations. We empirically compare the proposed model against other recent approaches on various conversational datasets. The results show that VMED consistently achieves significant improvement over others in both metric-based and qualitative evaluations.