38.4CLApr 23
Misinformation Span Detection in Videos via Audio TranscriptsBreno Matos, Rennan C. Lima, Savvas Zannettou et al.
Online misinformation is one of the most challenging issues lately, yielding severe consequences, including political polarization, attacks on democracy, and public health risks. Misinformation manifests in any platform with a large user base, including online social networks and messaging apps. It permeates all media and content forms, including images, text, audio, and video. Distinctly, video-based misinformation represents a multifaceted challenge for fact-checkers, given the ease with which individuals can record and upload videos on various video-sharing platforms. Previous research efforts investigated detecting video-based misinformation, focusing on whether a video shares misinformation or not on a video level. While this approach is useful, it only provides a limited and non-easily interpretable view of the problem given that it does not provide an additional context of when misinformation occurs within videos and what content (i.e., claims) are responsible for the video's misinformation nature. In this work, we attempt to bridge this research gap by creating two novel datasets that allow us to explore misinformation detection on videos via audio transcripts, focusing on identifying the span of videos that are responsible for the video's misinformation claim (misinformation span detection). We present two new datasets for this task. We transcribe each video's audio to text, identifying the video segment in which the misinformation claims appears, resulting in two datasets of more than 500 videos with over 2,400 segments containing annotated fact-checked claims. Then, we employ classifiers built with state-of-the-art language models, and our results show that we can identify in which part of a video there is misinformation with an F1 score of 0.68. We make publicly available our annotated datasets. We also release all transcripts, audio and videos.
29.4SIMar 25
WhatsApp Vaccine Discourse (WhaVax): An Expert-Annotated Dataset and Benchmark for Health Misinformation DetectionJônatas H. dos Santos, Julio C. S. Reis, Philipe Melo et al.
We introduce WhaVax, a new expert-annotated dataset of vaccine-related WhatsApp messages collected from large Brazilian public groups spanning multiple pandemic years. The dataset was constructed through a rigorous, carefully designed pipeline that integrates keyword-based data collection, semantic deduplication to remove near-duplicate content, and a multi-stage annotation protocol conducted by medical specialists. This process produced a high-quality gold-standard corpus, characterized by substantial inter-annotator agreement and strong reliability for downstream analysis. Additionally, we provide a detailed characterization of WhatsApp misinformation, revealing distinctive linguistic, structural, lexical, temporal, and group-level patterns, as well as a meaningful layer of ambiguous cases that reflect the complexity of health discourse in private messaging. We also benchmark classical models, fine-tuned Small Language Models, and zero- or few-shot Large Language Models under realistic data-scarcity constraints, demonstrating that strong embeddings and LLM approaches perform competitively, while domain alignment and data availability remain critical factors. This study provides a rare, high-quality resource to support misinformation research and computational modeling in encrypted communication environments.
SIMay 13, 2020
Neutrality May Matter: Sentiment Analysis in Reviews of Airbnb, Booking, and Couchsurfing in Brazil and USAGustavo Santos, Vinicius F. S. Mota, Fabricio Benevenuto et al.
Information and communications technologies have enabled the rise of the phenomenon named sharing economy, which represents activities between people, coordinated by online platforms, to obtain, provide, or share access to goods and services. In hosting services of the sharing economy, it is common to have a personal contact between the host and guest, and this may affect users' decision to do negative reviews, as negative reviews can damage the offered services. To evaluate this issue, we collected reviews from two sharing economy platforms, Airbnb and Couchsurfing, and from one platform that works mostly with hotels (traditional economy), Booking.com, for some cities in Brazil and the USA. Trough a sentiment analysis, we found that reviews in the sharing economy tend to be considerably more positive than those in the traditional economy. This can represent a problem in those systems, as an experiment with volunteers performed in this study suggests. In addition, we discuss how to exploit the results obtained to help improve users' decision making.