Beyond Semantics: Measuring Fine-Grained Emotion Preservation in Small Language Model-Based Machine Translation
For MT researchers and practitioners, this work addresses the underexplored problem of emotional fidelity in translation, but the findings are incremental as they apply existing models and prompting techniques to a new task.
The paper evaluates three Small Language Models (EuroLLM, Aya Expanse, Gemma) on preserving fine-grained emotions during backtranslation across five European languages using the GoEmotions dataset, finding that emotion-aware prompting improves preservation and ModernBERT outperforms BERT for emotion classification.
Preserving affective nuance remains a challenge in Machine Translation (MT), where semantic equivalence often takes precedence over emotional fidelity. This paper evaluates the performance of three state-of-the-art Small Language Models (SLMs) -- EuroLLM, Aya Expanse, and Gemma -- in maintaining fine-grained emotions during backtranslation. Using the GoEmotions dataset, which comprises Reddit comments across 28 distinct categories, we assess emotional preservation across five European languages: German, French, Spanish, Italian, and Polish. Specifically, we investigate (i) the inherent capability of these SLMs to retain emotional sentiment, (ii) the efficacy of emotion-aware prompting in improving preservation, and (iii) the performance of ModernBERT as a contemporary alternative to BERT for emotion classification in MT evaluation.