Old wine in old glasses: Comparing computational and qualitative methods in identifying incivility on Persian Twitter during the #MahsaAmini movement
This study addresses the problem of analyzing hate speech in low-resource languages like Persian for researchers and practitioners, but it is incremental as it compares existing methods without introducing new techniques.
The paper compared human coding, supervised learning with ParsBERT, and large language models (ChatGPT) for detecting incivility in Persian tweets from the #MahsaAmini movement, finding that ParsBERT substantially outperformed ChatGPT models in accuracy.
This paper compares three approaches to detecting incivility in Persian tweets: human qualitative coding, supervised learning with ParsBERT, and large language models (ChatGPT). Using 47,278 tweets from the #MahsaAmini movement in Iran, we evaluate the accuracy and efficiency of each method. ParsBERT substantially outperforms seven evaluated ChatGPT models in identifying hate speech. We also find that ChatGPT struggles not only with subtle cases but also with explicitly uncivil content, and that prompt language (English vs. Persian) does not meaningfully affect its outputs. The study provides a detailed comparison of these approaches and clarifies their strengths and limitations for analyzing hate speech in a low-resource language context.