Mix-and-Match: Scalable Dialog Response Retrieval using Gaussian Mixture Embeddings
This addresses the challenge of balancing scalability and modeling complex relationships in dialog systems, which is incremental as it combines existing approaches.
The paper tackles the problem of scalable dialog response retrieval by proposing a model that maps contexts and responses to probability distributions over embedding space, achieving better performance compared to other embedding-based approaches on publicly available conversation data.
Embedding-based approaches for dialog response retrieval embed the context-response pairs as points in the embedding space. These approaches are scalable, but fail to account for the complex, many-to-many relationships that exist between context-response pairs. On the other end of the spectrum, there are approaches that feed the context-response pairs jointly through multiple layers of neural networks. These approaches can model the complex relationships between context-response pairs, but fail to scale when the set of responses is moderately large (>100). In this paper, we combine the best of both worlds by proposing a scalable model that can learn complex relationships between context-response pairs. Specifically, the model maps the contexts as well as responses to probability distributions over the embedding space. We train the models by optimizing the Kullback-Leibler divergence between the distributions induced by context-response pairs in the training data. We show that the resultant model achieves better performance as compared to other embedding-based approaches on publicly available conversation data.