Neural Reranking Improves Subjective Quality of Machine Translation: NAIST at WAT2015
This work addresses translation quality for users by confirming neural reranking benefits extend to human judgment, though it is incremental as it builds on prior objective findings.
The paper tackled improving machine translation quality by adding a neural reranking component to syntax-based statistical models, finding that it significantly boosts both objective BLEU scores and subjective manual evaluations, with analysis showing gains primarily from enhanced grammatical correctness rather than lexical choice.
This year, the Nara Institute of Science and Technology (NAIST)'s submission to the 2015 Workshop on Asian Translation was based on syntax-based statistical machine translation, with the addition of a reranking component using neural attentional machine translation models. Experiments re-confirmed results from previous work stating that neural MT reranking provides a large gain in objective evaluation measures such as BLEU, and also confirmed for the first time that these results also carry over to manual evaluation. We further perform a detailed analysis of reasons for this increase, finding that the main contributions of the neural models lie in improvement of the grammatical correctness of the output, as opposed to improvements in lexical choice of content words.