Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training
This addresses the issue of dull and repetitive outputs in dialogue generation for users of conversational AI, though it appears incremental as it builds on existing adversarial training methods.
The paper tackled the problem of generic responses in neural dialogue systems by proposing Inverse Adversarial Training (IAT), which encourages sensitivity to perturbations in dialogue history to generate more diverse and consistent responses, with experimental results showing improved modeling of dialogue history on two benchmark datasets.
In this paper, we propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better. In contrast to standard adversarial training algorithms, IAT encourages the model to be sensitive to the perturbation in the dialogue history and therefore learning from perturbations. By giving higher rewards for responses whose output probability reduces more significantly when dialogue history is perturbed, the model is encouraged to generate more diverse and consistent responses. By penalizing the model when generating the same response given perturbed dialogue history, the model is forced to better capture dialogue history and generate more informative responses. Experimental results on two benchmark datasets show that our approach can better model dialogue history and generate more diverse and consistent responses. In addition, we point out a problem of the widely used maximum mutual information (MMI) based methods for improving the diversity of dialogue response generation models and demonstrate it empirically.