FarExStance: Explainable Stance Detection for Farsi
This work addresses stance detection for Farsi speakers, but it is incremental as it applies existing methods to a new language and dataset.
The authors tackled stance detection and explanation generation in Farsi by introducing the FarExStance dataset and evaluating models like fine-tuned RoBERTa and LLMs, finding that fine-tuned RoBERTa and Aya-23-8B performed best on stance detection, while GPT-4o and Claude-3.5-Sonnet led in explanation quality based on different metrics.
We introduce FarExStance, a new dataset for explainable stance detection in Farsi. Each instance in this dataset contains a claim, the stance of an article or social media post towards that claim, and an extractive explanation which provides evidence for the stance label. We compare the performance of a fine-tuned multilingual RoBERTa model to several large language models in zero-shot, few-shot, and parameter-efficient fine-tuned settings on our new dataset. On stance detection, the most accurate models are the fine-tuned RoBERTa model, the LLM Aya-23-8B which has been fine-tuned using parameter-efficient fine-tuning, and few-shot Claude-3.5-Sonnet. Regarding the quality of the explanations, our automatic evaluation metrics indicate that few-shot GPT-4o generates the most coherent explanations, while our human evaluation reveals that the best Overall Explanation Score (OES) belongs to few-shot Claude-3.5-Sonnet. The fine-tuned Aya-32-8B model produced explanations most closely aligned with the reference explanations.