Cued@wmt19:ewc&lms
This work addresses incremental improvements in machine translation for researchers and practitioners in the field.
The paper tackled the problem of improving machine translation performance in the WMT19 evaluation by fine-tuning strong baselines with elastic weight consolidation and language modeling techniques, achieving substantial gains through checkpoint averaging and novel LM architectures.
Two techniques provide the fabric of the Cambridge University Engineering Department's (CUED) entry to the WMT19 evaluation campaign: elastic weight consolidation (EWC) and different forms of language modelling (LMs). We report substantial gains by fine-tuning very strong baselines on former WMT test sets using a combination of checkpoint averaging and EWC. A sentence-level Transformer LM and a document-level LM based on a modified Transformer architecture yield further gains. As in previous years, we also extract $n$-gram probabilities from SMT lattices which can be seen as a source-conditioned $n$-gram LM.