End-to-end Adversarial Learning for Generative Conversational Agents
This work addresses the challenge of generating more human-like and diverse responses in conversational AI, though it is incremental as it builds on existing adversarial approaches for dialogue generation.
The paper tackles the problem of training generative conversational agents by introducing an adversarial learning method with token-level classification, enabling end-to-end training and improving performance on out-of-distribution questions, with experimental results showing significant gains over teacher forcing training.
This paper presents a new adversarial learning method for generative conversational agents (GCA) besides a new model of GCA. Similar to previous works on adversarial learning for dialogue generation, our method assumes the GCA as a generator that aims at fooling a discriminator that labels dialogues as human-generated or machine-generated; however, in our approach, the discriminator performs token-level classification, i.e. it indicates whether the current token was generated by humans or machines. To do so, the discriminator also receives the context utterances (the dialogue history) and the incomplete answer up to the current token as input. This new approach makes possible the end-to-end training by backpropagation. A self-conversation process enables to produce a set of generated data with more diversity for the adversarial training. This approach improves the performance on questions not related to the training data. Experimental results with human and adversarial evaluations show that the adversarial method yields significant performance gains over the usual teacher forcing training.