Social Perceptions of English Spelling Variation on Twitter: A Comparative Analysis of Human and LLM Responses
This work addresses how spelling variation influences social perception in online writing, with implications for sociolinguistics and AI alignment, though it is incremental in comparing existing methods.
The study tackled the problem of understanding social perceptions of English spelling variation on Twitter by comparing human and large language model (LLM) ratings on attributes like formality, carefulness, and age, finding generally strong correlations but notable differences in rating distributions and variation types.
Spelling variation (e.g. funnnn vs. fun) can influence the social perception of texts and their writers: we often have various associations with different forms of writing (is the text informal? does the writer seem young?). In this study, we focus on the social perception of spelling variation in online writing in English and study to what extent this perception is aligned between humans and large language models (LLMs). Building on sociolinguistic methodology, we compare LLM and human ratings on three key social attributes of spelling variation (formality, carefulness, age). We find generally strong correlations in the ratings between humans and LLMs. However, notable differences emerge when we analyze the distribution of ratings and when comparing between different types of spelling variation.