MTQ-Eval: Multilingual Text Quality Evaluation for Language Models
This addresses the need for scalable and effective multilingual text quality evaluation for language model users, though it appears incremental as it builds on existing LLM-based evaluation methods.
The authors tackled the problem of evaluating text quality in multilingual contexts by introducing MTQ-Eval, a framework that learns from high- and low-quality text examples, resulting in improved performance across 115 languages and downstream tasks.
The use of large language models (LLMs) for evaluating outputs is becoming an increasingly effective and scalable approach. However, it remains uncertain whether this capability extends beyond task-specific evaluations to more general assessments of text quality, particularly in multilingual contexts. In this study, we introduce, MTQ-Eval, a novel framework for multilingual text quality evaluation that learns from examples of both high- and low-quality texts, adjusting its internal representations. To develop MTQ-Eval, we first automatically generate text quality preference data and then use it to train open-source base LLMs to align with ratings of high- and low-quality text. Our comprehensive evaluation across 115 languages demonstrates the improved performance of the proposed model. Upon further analysis, we find that this enhanced evaluation capability also leads to notable improvements in downstream tasks.