TruthStance: An Annotated Dataset of Conversations on Truth Social
This provides a resource for researchers studying online discourse on under-studied platforms, though it is incremental as it extends existing methods to new data.
The authors tackled the lack of datasets for argument mining and stance detection on alt-tech platforms by introducing TruthStance, a large-scale annotated dataset from Truth Social with 24,378 posts and 523,360 comments, and used it to evaluate LLM prompting strategies, releasing additional LLM-generated labels for thousands of instances.
Argument mining and stance detection are central to understanding how opinions are formed and contested in online discourse. However, most publicly available resources focus on mainstream platforms such as Twitter and Reddit, leaving conversational structure on alt-tech platforms comparatively under-studied. We introduce TruthStance, a large-scale dataset of Truth Social conversation threads spanning 2023-2025, consisting of 24,378 posts and 523,360 comments with reply-tree structure preserved. We provide a human-annotated benchmark of 1,500 instances across argument mining and claim-based stance detection, including inter-annotator agreement, and use it to evaluate large language model (LLM) prompting strategies. Using the best-performing configuration, we release additional LLM-generated labels for 24,352 posts (argument presence) and 107,873 comments (stance to parent), enabling analysis of stance and argumentation patterns across depth, topics, and users. All code and data are released publicly.