Open Machine Translation for Esperanto
This work addresses the underexplored problem of machine translation for Esperanto, providing a benchmark for researchers and the Esperanto community, but it is incremental as it applies existing methods to a new language.
The authors conducted the first comprehensive evaluation of open-source machine translation systems for Esperanto, comparing rule-based systems, encoder-decoder models, and LLMs across six language directions involving English, Spanish, Catalan, and Esperanto. They found that the NLLB family achieved the best performance, with human evaluation preferring NLLB translations in approximately half of comparisons, though errors persisted.
Esperanto is a widespread constructed language, known for its regular grammar and productive word formation. Besides having substantial resources available thanks to its online community, it remains relatively underexplored in the context of modern machine translation (MT) approaches. In this work, we present the first comprehensive evaluation of open-source MT systems for Esperanto, comparing rule-based systems, encoder-decoder models, and LLMs across model sizes. We evaluate translation quality across six language directions involving English, Spanish, Catalan, and Esperanto using multiple automatic metrics as well as human evaluation. Our results show that the NLLB family achieves the best performance in all language pairs, followed closely by our trained compact models and a fine-tuned general-purpose LLM. Human evaluation confirms this trend, with NLLB translations preferred in approximately half of the comparisons, although noticeable errors remain. In line with Esperanto's tradition of openness and international collaboration, we release our code and best-performing models publicly.