Dynamic Integration of Background Knowledge in Neural NLU Systems
This addresses the limitation of static knowledge in NLU systems, potentially improving performance in tasks requiring common-sense reasoning, though it appears incremental as it builds on existing NLU architectures.
The paper tackles the problem of integrating explicit background knowledge into neural natural language understanding systems, introducing a new architecture that dynamically incorporates free-text knowledge statements, and demonstrates effectiveness with experiments on document question answering and recognizing textual entailment.
Common-sense and background knowledge is required to understand natural language, but in most neural natural language understanding (NLU) systems, this knowledge must be acquired from training corpora during learning, and then it is static at test time. We introduce a new architecture for the dynamic integration of explicit background knowledge in NLU models. A general-purpose reading module reads background knowledge in the form of free-text statements (together with task-specific text inputs) and yields refined word representations to a task-specific NLU architecture that reprocesses the task inputs with these representations. Experiments on document question answering (DQA) and recognizing textual entailment (RTE) demonstrate the effectiveness and flexibility of the approach. Analysis shows that our model learns to exploit knowledge in a semantically appropriate way.