Recurrent Graph Syntax Encoder for Neural Machine Translation
This work addresses the challenge of better reasoning and meaning preservation in machine translation for NLP applications, representing an incremental improvement over existing syntax-aware models.
The paper tackles the problem of improving neural machine translation by incorporating syntactic information, proposing a Recurrent Graph Syntax Encoder (RGSE) that enhances syntax capture and achieves considerable improvements in English-to-German and English-to-Czech tasks, with competitive results against state-of-the-art models in WMT14 En-De.
Syntax-incorporated machine translation models have been proven successful in improving the model's reasoning and meaning preservation ability. In this paper, we propose a simple yet effective graph-structured encoder, the Recurrent Graph Syntax Encoder, dubbed \textbf{RGSE}, which enhances the ability to capture useful syntactic information. The RGSE is done over a standard encoder (recurrent or self-attention encoder), regarding recurrent network units as graph nodes and injects syntactic dependencies as edges, such that RGSE models syntactic dependencies and sequential information (\textit{i.e.}, word order) simultaneously. Our approach achieves considerable improvements over several syntax-aware NMT models in English$\Rightarrow$German and English$\Rightarrow$Czech translation tasks. And RGSE-equipped big model obtains competitive result compared with the state-of-the-art model in WMT14 En-De task. Extensive analysis further verifies that RGSE could benefit long sentence modeling, and produces better translations.