Lessons from the Bible on Modern Topics: Low-Resource Multilingual Topic Model Evaluation
This work addresses a critical evaluation gap for researchers and practitioners using multilingual topic models, particularly for low-resource languages, though it is incremental as it builds on existing metrics.
The authors tackled the lack of a standard metric for evaluating multilingual topic models, especially in low-resource languages, by introducing a new intrinsic evaluation method that correlates well with human judgments and downstream performance, and proposing an adaptation model to improve metric accuracy in low-resource settings.
Multilingual topic models enable document analysis across languages through coherent multilingual summaries of the data. However, there is no standard and effective metric to evaluate the quality of multilingual topics. We introduce a new intrinsic evaluation of multilingual topic models that correlates well with human judgments of multilingual topic coherence as well as performance in downstream applications. Importantly, we also study evaluation for low-resource languages. Because standard metrics fail to accurately measure topic quality when robust external resources are unavailable, we propose an adaptation model that improves the accuracy and reliability of these metrics in low-resource settings.