CAT-Translate: Building Compact Open-Source Models for Japanese-English Translation
For developers needing Japanese-English translation, this work shows that compact specialized models can surpass large multilingual models in practical scenarios, challenging the assumption that larger models are always better.
The authors developed compact open-source models (0.8B–7B parameters) specialized for Japanese-English translation using two-stage supervised fine-tuning and Multi-Objective GRPO. Their models outperform large multilingual models on real-world benchmarks across multiple domains, demonstrating the value of specialized models.
Nowadays, large multilingual translation models demonstrate impressive translation capabilities in the machine translation benchmarks. This raises a practical question to the developers: is it worth developing translation models specialized for a particular language pair if you only need to support that language pair? To give an anecdotal answer to this question, we develop a family of small language models (0.8B, 1.4B, 3.3B, and 7B parameters) specialized for Japanese-English bidirectional translation. We employ a two-stage supervised fine-tuning approach followed by Multi-Objective GRPO (Ichihara et al. 2025) to train models on synthetically generated parallel corpora. We evaluate our models on WMT and real-world translation benchmarks across business, legal, medical, financial, and patent domains. While multilingual models achieve strong performance on WMT benchmarks, our compact models outperform them on real-world benchmarks, suggesting the practical utility of developing specialized translation models even in the era of large multilingual models.