Transfer Learning from Transformers to Fake News Challenge Stance Detection (FNC-1) Task
This work addresses fake news detection for media and fact-checking applications, but it is incremental as it applies existing methods to a specific dataset.
The paper tackled the Fake News Challenge Stage 1 stance detection task by leveraging Transformer-based language models like BERT, XLNet, and RoBERTa, achieving state-of-the-art results through fine-tuning and feature integration.
In this paper, we report improved results of the Fake News Challenge Stage 1 (FNC-1) stance detection task. This gain in performance is due to the generalization power of large language models based on Transformer architecture, invented, trained and publicly released over the last two years. Specifically (1) we improved the FNC-1 best performing model adding BERT sentence embedding of input sequences as a model feature, (2) we fine-tuned BERT, XLNet, and RoBERTa transformers on FNC-1 extended dataset and obtained state-of-the-art results on FNC-1 task.