A Discrete CVAE for Response Generation on Short-Text Conversation
This work addresses the issue of low-quality and non-diverse responses in chatbots and conversational AI systems, representing an incremental improvement over existing CVAE-based methods.
The paper tackled the problem of generating bland and generic responses in neural conversation models by introducing a discrete latent variable with explicit semantic meaning into a conditional variational autoencoder (CVAE) for short-text conversation, resulting in improved performance over various generation models in automatic and human evaluations, with more diverse and informative responses.
Neural conversation models such as encoder-decoder models are easy to generate bland and generic responses. Some researchers propose to use the conditional variational autoencoder(CVAE) which maximizes the lower bound on the conditional log-likelihood on a continuous latent variable. With different sampled la-tent variables, the model is expected to generate diverse responses. Although the CVAE-based models have shown tremendous potential, their improvement of generating high-quality responses is still unsatisfactory. In this paper, we introduce a discrete latent variable with an explicit semantic meaning to improve the CVAE on short-text conversation. A major advantage of our model is that we can exploit the semantic distance between the latent variables to maintain good diversity between the sampled latent variables. Accordingly, we pro-pose a two-stage sampling approach to enable efficient diverse variable selection from a large latent space assumed in the short-text conversation task. Experimental results indicate that our model outperforms various kinds of generation models under both automatic and human evaluations and generates more diverse and in-formative responses.