EM Pre-training for Multi-party Dialogue Response Generation
This work addresses the challenge of multi-party dialogue response generation for AI systems, but it is incremental as it adapts existing EM methods to a specific data limitation.
The paper tackles the problem of generating responses in multi-party dialogues by addressing the lack of annotated addressee labels, proposing an Expectation-Maximization (EM) approach that iteratively generates labels and optimizes the model, with experiments justifying its feasibility and effectiveness.
Dialogue response generation requires an agent to generate a response according to the current dialogue history, in terms of which two-party dialogues have been well studied, but leaving a great gap for multi-party dialogues at the same time. Different from two-party dialogues where each response is a direct reply to its previous utterance, the addressee of a response utterance should be specified before it is generated in the multi-party scenario. Thanks to the huge amount of two-party conversational data, various pre-trained language models for two-party dialogue response generation have been proposed. However, due to the lack of annotated addressee labels in multi-party dialogue datasets, it is hard to use them to pre-train a response generation model for multi-party dialogues. To tackle this obstacle, we propose an Expectation-Maximization (EM) approach that iteratively performs the expectation steps to generate addressee labels, and the maximization steps to optimize a response generation model. Theoretical analyses and extensive experiments have justified the feasibility and effectiveness of our proposed method.