Token-free Models for Sarcasm Detection
This work addresses sarcasm detection for NLP applications in noisy domains like social media, but it is incremental as it applies existing token-free models to a specific task.
The paper tackled sarcasm detection by evaluating token-free models like ByT5 and CANINE against token-based baselines, achieving new state-of-the-art performance with accuracy improvements of 0.77% on news headlines and 0.49% on Twitter datasets.
Tokenization is a foundational step in most natural language processing (NLP) pipelines, yet it introduces challenges such as vocabulary mismatch and out-of-vocabulary issues. Recent work has shown that models operating directly on raw text at the byte or character level can mitigate these limitations. In this paper, we evaluate two token-free models, ByT5 and CANINE, on the task of sarcasm detection in both social media (Twitter) and non-social media (news headlines) domains. We fine-tune and benchmark these models against token-based baselines and state-of-the-art approaches. Our results show that ByT5-small and CANINE outperform token-based counterparts and achieve new state-of-the-art performance, improving accuracy by 0.77% and 0.49% on the News Headlines and Twitter Sarcasm datasets, respectively. These findings underscore the potential of token-free models for robust NLP in noisy and informal domains such as social media.