Sarc7: Evaluating Sarcasm Detection and Generation with Seven Types and Emotion-Informed Techniques
This work addresses sarcasm interpretation for natural language processing, but it is incremental as it builds on existing datasets and methods.
The paper tackled sarcasm detection and generation by introducing Sarc7, a benchmark with 7 sarcasm types, and found that emotion-based prompting with Gemini 2.5 achieved an F1 score of 0.3664 and 38.46% more successful generations than zero-shot prompting.
Sarcasm is a form of humor where expressions convey meanings opposite to their literal interpretations. Classifying and generating sarcasm using large language models is vital for interpreting human communication. Sarcasm poses challenges for computational models, due to its nuanced nature. We introduce Sarc7, a benchmark that classifies 7 types of sarcasm: self-deprecating, brooding, deadpan, polite, obnoxious, raging, and manic by annotating entries of the MUStARD dataset. Classification was evaluated using zero-shot, few-shot, chain-of-thought (CoT), and a novel emotion-based prompting technique. We propose an emotion-based generation method developed by identifying key components of sarcasm-incongruity, shock value, and context dependency. Our classification experiments show that Gemini 2.5, using emotion-based prompting, outperforms other setups with an F1 score of 0.3664. Human evaluators preferred our emotion-based prompting, with 38.46% more successful generations than zero-shot prompting.