Overprotective Training Environments Fall Short at Testing Time: Let Models Contribute to Their Own Training
This addresses the issue of unnatural dialogue generation in conversational AI systems, but it is incremental as it builds on existing training paradigms.
The paper tackles the problem of conversational systems generating unnatural dialogues due to mismatched training and testing conditions, proposing a method that trains models with mixed batches of human and machine-generated dialogues, and validates it on the GuessWhat?! visual referential game.
Despite important progress, conversational systems often generate dialogues that sound unnatural to humans. We conjecture that the reason lies in their different training and testing conditions: agents are trained in a controlled "lab" setting but tested in the "wild". During training, they learn to generate an utterance given the human dialogue history. On the other hand, during testing, they must interact with each other, and hence deal with noisy data. We propose to fill this gap by training the model with mixed batches containing both samples of human and machine-generated dialogues. We assess the validity of the proposed method on GuessWhat?!, a visual referential game.