An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation
This addresses the problem of limited diversity in lightweight dialogue systems for users needing accessible AI interactions, though it is incremental as it builds on existing multi-decoder methods.
The paper tackles the challenge of generating diverse dialogue responses with medium-to-small-sized models by proposing an Equal-size Hard EM algorithm, which trains a multi-decoder model and achieves high-quality diverse responses as verified on two large-scale datasets.
Open-domain dialogue systems aim to interact with humans through natural language texts in an open-ended fashion. Despite the recent success of super large dialogue systems such as ChatGPT, using medium-to-small-sized dialogue systems remains the common practice as they are more lightweight and accessible; however, generating diverse dialogue responses is challenging, especially with smaller models. In this work, we propose an Equal-size Hard Expectation--Maximization (EqHard-EM) algorithm to train a multi-decoder model for diverse dialogue generation. Our algorithm assigns a sample to a decoder in a hard manner and additionally imposes an equal-assignment constraint to ensure that all decoders are well-trained. We provide detailed theoretical analysis to justify our approach. Further, experiments on two large-scale open-domain dialogue datasets verify that our EqHard-EM algorithm generates high-quality diverse responses.