Leveraging Knowledge and Reinforcement Learning for Enhanced Reliability of Language Models
This addresses reliability issues in language models for NLP practitioners, though it appears incremental as it builds on existing ensembling and knowledge integration techniques.
The paper tackles the overlooked reliability problem in language models by developing a knowledge-guided ensembling approach using reinforcement learning to integrate ConceptNet and Wikipedia knowledge, which improves both reliability and accuracy scores across nine GLUE datasets and outperforms state-of-the-art methods.
The Natural Language Processing(NLP) community has been using crowd sourcing techniques to create benchmark datasets such as General Language Understanding and Evaluation(GLUE) for training modern Language Models such as BERT. GLUE tasks measure the reliability scores using inter annotator metrics i.e. Cohens Kappa. However, the reliability aspect of LMs has often been overlooked. To counter this problem, we explore a knowledge-guided LM ensembling approach that leverages reinforcement learning to integrate knowledge from ConceptNet and Wikipedia as knowledge graph embeddings. This approach mimics human annotators resorting to external knowledge to compensate for information deficits in the datasets. Across nine GLUE datasets, our research shows that ensembling strengthens reliability and accuracy scores, outperforming state of the art.