Does Knowledge Help General NLU? An Empirical Study
This work addresses the problem of enhancing general NLU for NLP practitioners by providing insights into knowledge integration, though it is incremental as it builds on existing methods.
The study empirically investigates whether integrating external knowledge into language models improves performance on general natural language understanding tasks, finding that it significantly boosts results on certain tasks without harming others.
It is often observed in knowledge-centric tasks (e.g., common sense question and answering, relation classification) that the integration of external knowledge such as entity representation into language models can help provide useful information to boost the performance. However, it is still unclear whether this benefit can extend to general natural language understanding (NLU) tasks. In this work, we empirically investigated the contribution of external knowledge by measuring the end-to-end performance of language models with various knowledge integration methods. We find that the introduction of knowledge can significantly improve the results on certain tasks while having no adverse effects on other tasks. We then employ mutual information to reflect the difference brought by knowledge and a neural interpretation model to reveal how a language model utilizes external knowledge. Our study provides valuable insights and guidance for practitioners to equip NLP models with knowledge.