Autoencoding Language Model Based Ensemble Learning for Commonsense Validation and Explanation
This addresses the problem of building AI systems that understand human commonsense, though it is incremental as it combines existing models for a specific task.
The paper tackles commonsense validation and explanation by proposing ALMEn, an ensemble method using autoencoding language models and Siamese networks, achieving 97.9% accuracy on validation and 95.4% on explanation selection on the SemEval-2020 benchmark.
An ultimate goal of artificial intelligence is to build computer systems that can understand human languages. Understanding commonsense knowledge about the world expressed in text is one of the foundational and challenging problems to create such intelligent systems. As a step towards this goal, we present in this paper ALMEn, an Autoencoding Language Model based Ensemble learning method for commonsense validation and explanation. By ensembling several advanced pre-trained language models including RoBERTa, DeBERTa, and ELECTRA with Siamese neural networks, our method can distinguish natural language statements that are against commonsense (validation subtask) and correctly identify the reason for making against commonsense (explanation selection subtask). Experimental results on the benchmark dataset of SemEval-2020 Task 4 show that our method outperforms state-of-the-art models, reaching 97.9% and 95.4% accuracies on the validation and explanation selection subtasks, respectively.