DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning
This work addresses the problem of modeling annotator perspectives in NLP tasks, but it is incremental as it applies existing methods to a shared task without major breakthroughs.
The paper tackled predicting annotator-specific annotations in perspectivist tasks by exploring in-context learning with large language models and label distribution learning with RoBERTa, showing that aggregating ICL predictions into soft labels yields competitive performance.
This system paper presents the DeMeVa team's approaches to the third edition of the Learning with Disagreements shared task (LeWiDi 2025; Leonardelli et al., 2025). We explore two directions: in-context learning (ICL) with large language models, where we compare example sampling strategies; and label distribution learning (LDL) methods with RoBERTa (Liu et al., 2019b), where we evaluate several fine-tuning methods. Our contributions are twofold: (1) we show that ICL can effectively predict annotator-specific annotations (perspectivist annotations), and that aggregating these predictions into soft labels yields competitive performance; and (2) we argue that LDL methods are promising for soft label predictions and merit further exploration by the perspectivist community.