Improving Question Answering by Commonsense-Based Pre-Training
This addresses a key limitation in NLP for tasks needing commonsense, though it is incremental as it builds on existing models.
The paper tackled the problem of neural networks struggling with commonsense question answering by pre-training relational functions using ConceptNet, resulting in improved baseline performance on three QA tasks requiring commonsense reasoning.
Although neural network approaches achieve remarkable success on a variety of NLP tasks, many of them struggle to answer questions that require commonsense knowledge. We believe the main reason is the lack of commonsense \mbox{connections} between concepts. To remedy this, we provide a simple and effective method that leverages external commonsense knowledge base such as ConceptNet. We pre-train direct and indirect relational functions between concepts, and show that these pre-trained functions could be easily added to existing neural network models. Results show that incorporating commonsense-based function improves the baseline on three question answering tasks that require commonsense reasoning. Further analysis shows that our system \mbox{discovers} and leverages useful evidence from an external commonsense knowledge base, which is missing in existing neural network models and help derive the correct answer.