Can Large Language Models Address Open-Target Stance Detection?
This work addresses a more realistic stance detection task for NLP applications, though it is incremental as it builds on existing methods with new evaluations.
The paper tackles the problem of Open-Target Stance Detection (OTSD), where targets are unseen during training and not provided as input, by evaluating Large Language Models (LLMs) against the existing Target-Stance Extraction (TSE) method. The results show that LLMs outperform TSE in target generation and stance detection when targets are explicit, but struggle when targets are not explicit.
Stance detection (SD) identifies the text position towards a target, typically labeled as favor, against, or none. We introduce Open-Target Stance Detection (OTSD), the most realistic task where targets are neither seen during training nor provided as input. We evaluate Large Language Models (LLMs) from GPT, Gemini, Llama, and Mistral families, comparing their performance to the only existing work, Target-Stance Extraction (TSE), which benefits from predefined targets. Unlike TSE, OTSD removes the dependency of a predefined list, making target generation and evaluation more challenging. We also provide a metric for evaluating target quality that correlates well with human judgment. Our experiments reveal that LLMs outperform TSE in target generation, both when the real target is explicitly and not explicitly mentioned in the text. Similarly, LLMs overall surpass TSE in stance detection for both explicit and non-explicit cases. However, LLMs struggle in both target generation and stance detection when the target is not explicit.