LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits based Humor Grading
This work addresses humor grading in NLP, but it is incremental as it applies existing models to new datasets without introducing novel methods.
The paper assessed BERT and its derivatives (RoBERTa, DistilBERT, ALBERT) for humor grading on short-edits datasets, finding that these models demonstrated significant generalization capabilities in tasks like zero-shot and cross-dataset inference.
In this paper, we assess the ability of BERT and its derivative models (RoBERTa, DistilBERT, and ALBERT) for short-edits based humor grading. We test these models for humor grading and classification tasks on the Humicroedit and the FunLines dataset. We perform extensive experiments with these models to test their language modeling and generalization abilities via zero-shot inference and cross-dataset inference based approaches. Further, we also inspect the role of self-attention layers in humor-grading by performing a qualitative analysis over the self-attention weights from the final layer of the trained BERT model. Our experiments show that all the pre-trained BERT derivative models show significant generalization capabilities for humor-grading related tasks.