An Analysis of Source-Side Grammatical Errors in NMT
This addresses the issue of NMT robustness for users dealing with noisy input, but it is incremental as it focuses on analysis rather than a new solution.
The study tackled the problem of Neural Machine Translation (NMT) quality degradation due to source-side grammatical errors by evaluating English-to-German NMT on real grammatical noise, finding significant degradation and introducing methods for robustness evaluation without true references.
The quality of Neural Machine Translation (NMT) has been shown to significantly degrade when confronted with source-side noise. We present the first large-scale study of state-of-the-art English-to-German NMT on real grammatical noise, by evaluating on several Grammar Correction corpora. We present methods for evaluating NMT robustness without true references, and we use them for extensive analysis of the effects that different grammatical errors have on the NMT output. We also introduce a technique for visualizing the divergence distribution caused by a source-side error, which allows for additional insights.