Boosting Neural Machine Translation
This addresses training efficiency for NMT researchers and practitioners, but it is incremental as it builds on existing methods without network modifications.
The paper tackles the problem of high computational cost and slow training in Neural Machine Translation by proposing data boosting and bootstrap methods that mimic human learning, resulting in up to 1.63 BLEU improvement and 20% training time savings.
Training efficiency is one of the main problems for Neural Machine Translation (NMT). Deep networks need for very large data as well as many training iterations to achieve state-of-the-art performance. This results in very high computation cost, slowing down research and industrialisation. In this paper, we propose to alleviate this problem with several training methods based on data boosting and bootstrap with no modifications to the neural network. It imitates the learning process of humans, which typically spend more time when learning "difficult" concepts than easier ones. We experiment on an English-French translation task showing accuracy improvements of up to 1.63 BLEU while saving 20% of training time.