A Survey on Multi-modal Machine Translation: Tasks, Methods and Challenges
It offers researchers an updated overview of the field's current state, but is incremental as it builds on existing literature without introducing new methods.
This paper provides a comprehensive survey of multi-modal machine translation, analyzing 99 prior works to summarize models, datasets, and evaluation metrics, while also discussing emerging types and future research directions.
In recent years, multi-modal machine translation has attracted significant interest in both academia and industry due to its superior performance. It takes both textual and visual modalities as inputs, leveraging visual context to tackle the ambiguities in source texts. In this paper, we begin by offering an exhaustive overview of 99 prior works, comprehensively summarizing representative studies from the perspectives of dominant models, datasets, and evaluation metrics. Afterwards, we analyze the impact of various factors on model performance and finally discuss the possible research directions for this task in the future. Over time, multi-modal machine translation has developed more types to meet diverse needs. Unlike previous surveys confined to the early stage of multi-modal machine translation, our survey thoroughly concludes these emerging types from different aspects, so as to provide researchers with a better understanding of its current state.