No Need to Pay Attention: Simple Recurrent Neural Networks Work! (for Answering "Simple" Questions)
This work addresses the problem of efficient and accurate question answering for industry applications like Comcast's X1 platform, though it is incremental as it builds on existing neural network approaches with a simpler design.
The paper tackled first-order factoid question answering by formulating it as entity detection and relation classification, using simple recurrent neural networks, and achieved substantial improvements over previous methods on the SimpleQuestions dataset, with concrete accuracy gains from 65%-76% to higher unspecified levels.
First-order factoid question answering assumes that the question can be answered by a single fact in a knowledge base (KB). While this does not seem like a challenging task, many recent attempts that apply either complex linguistic reasoning or deep neural networks achieve 65%-76% accuracy on benchmark sets. Our approach formulates the task as two machine learning problems: detecting the entities in the question, and classifying the question as one of the relation types in the KB. We train a recurrent neural network to solve each problem. On the SimpleQuestions dataset, our approach yields substantial improvements over previously published results --- even neural networks based on much more complex architectures. The simplicity of our approach also has practical advantages, such as efficiency and modularity, that are valuable especially in an industry setting. In fact, we present a preliminary analysis of the performance of our model on real queries from Comcast's X1 entertainment platform with millions of users every day.