mCPT at SemEval-2023 Task 3: Multilingual Label-Aware Contrastive Pre-Training of Transformers for Few- and Zero-shot Framing Detection
This work addresses the problem of detecting frames with limited data for computational social science, though it is incremental as it builds on existing multilingual Transformers.
The paper tackled multilingual few- and zero-shot framing detection by developing a system using label-aware contrastive pre-training of Transformers, which won the zero-shot Spanish task and achieved competitive results in eight other languages.
This paper presents the winning system for the zero-shot Spanish framing detection task, which also achieves competitive places in eight additional languages. The challenge of the framing detection task lies in identifying a set of 14 frames when only a few or zero samples are available, i.e., a multilingual multi-label few- or zero-shot setting. Our developed solution employs a pre-training procedure based on multilingual Transformers using a label-aware contrastive loss function. In addition to describing the system, we perform an embedding space analysis and ablation study to demonstrate how our pre-training procedure supports framing detection to advance computational framing analysis.