A Study of Slang Representation Methods
This work addresses the need for slang-aware tools to assist in moderating online content, but it is incremental as it builds on existing methods without introducing a new paradigm.
The study tackled the problem of slang understanding for social good tasks by evaluating combinations of representation learning models and knowledge resources, finding that models pre-trained on social media data performed best, with dictionaries only helping static word embeddings.
Considering the large amount of content created online by the minute, slang-aware automatic tools are critically needed to promote social good, and assist policymakers and moderators in restricting the spread of offensive language, abuse, and hate speech. Despite the success of large language models and the spontaneous emergence of slang dictionaries, it is unclear how far their combination goes in terms of slang understanding for downstream social good tasks. In this paper, we provide a framework to study different combinations of representation learning models and knowledge resources for a variety of downstream tasks that rely on slang understanding. Our experiments show the superiority of models that have been pre-trained on social media data, while the impact of dictionaries is positive only for static word embeddings. Our error analysis identifies core challenges for slang representation learning, including out-of-vocabulary words, polysemy, variance, and annotation disagreements, which can be traced to characteristics of slang as a quickly evolving and highly subjective language.