The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translation
This work clarifies the relative importance of vocabulary overlap versus language relatedness and domain match for practitioners building multilingual translation systems.
The paper investigates the role of vocabulary overlap in knowledge transfer for multilingual neural machine translation, finding that domain-match and language relatedness are more important than joint vocabulary for performance.
Knowledge transfer, especially across related languages, has been found beneficial for multilingual neural machine translation (MNMT), but some aspects are still under-explored and deserve further investigation. A joint vocabulary is most often applied to form a uniform word embedding space, but since the impact of a disjoint vocabulary on model performance is far less studied, there is no consensus on how much knowledge transfer is mainly due to vocabulary overlap. In this paper, we present systematic experiments with joint and disjoint vocabularies, and auxiliary languages related and unrelated to the source language. We design this experiment in an out-of-domain setup in order to emphasize transfer and the impact of the auxiliary language. As expected, we yield better results with more extensive vocabulary overlaps typical for related languages, but our experiments also show that domain-match and language relatedness are more important than a joint vocabulary.