Improving Computer Generated Dialog with Auxiliary Loss Functions and Custom Evaluation Metrics
This work addresses the challenge of creating more natural and engaging chatbots, but it is incremental as it builds on existing methods in neural network-based dialogue generation.
The authors tackled the problem of generating coherent and cohesive dialogue by enhancing a Recurrent Neural Network with auxiliary loss functions (Maximum Mutual Information and entropy) and beam search, resulting in improved performance as measured by custom evaluation metrics.
Although people have the ability to engage in vapid dialogue without effort, this may not be a uniquely human trait. Since the 1960's researchers have been trying to create agents that can generate artificial conversation. These programs are commonly known as chatbots. With increasing use of neural networks for dialog generation, some conclude that this goal has been achieved. This research joins the quest by creating a dialog generating Recurrent Neural Network (RNN) and by enhancing the ability of this network with auxiliary loss functions and a beam search. Our custom loss functions achieve better cohesion and coherence by including calculations of Maximum Mutual Information (MMI) and entropy. We demonstrate the effectiveness of this system by using a set of custom evaluation metrics inspired by an abundance of previous research and based on tried-and-true principles of Natural Language Processing.