LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model
This work addresses commonsense reasoning in NLP, but it is incremental as it applies existing models with minor enhancements to a specific competition task.
The paper tackled commonsense validation and explanation by using pretrained language models for two subtasks, achieving accuracy scores of 95.6 and 94.9 on the test set and ranking 7th and 2nd on the leaderboard.
This paper describes our submission to subtask a and b of SemEval-2020 Task 4. For subtask a, we use a ALBERT based model with improved input form to pick out the common sense statement from two statement candidates. For subtask b, we use a multiple choice model enhanced by hint sentence mechanism to select the reason from given options about why a statement is against common sense. Besides, we propose a novel transfer learning strategy between subtasks which help improve the performance. The accuracy scores of our system are 95.6 / 94.9 on official test set and rank 7$^{th}$ / 2$^{nd}$ on Post-Evaluation leaderboard.