Vocabulary Selection Strategies for Neural Machine Translation
This work addresses efficiency bottlenecks for neural machine translation systems, making them faster and more practical for deployment, though it is incremental as it builds on prior vocabulary selection methods.
The paper tackled the problem of improving efficiency in neural machine translation by experimenting with vocabulary selection strategies, showing that decoding time on CPUs can be reduced by up to 90% and training time by 25% with negligible accuracy loss on WMT tasks.
Classical translation models constrain the space of possible outputs by selecting a subset of translation rules based on the input sentence. Recent work on improving the efficiency of neural translation models adopted a similar strategy by restricting the output vocabulary to a subset of likely candidates given the source. In this paper we experiment with context and embedding-based selection methods and extend previous work by examining speed and accuracy trade-offs in more detail. We show that decoding time on CPUs can be reduced by up to 90% and training time by 25% on the WMT15 English-German and WMT16 English-Romanian tasks at the same or only negligible change in accuracy. This brings the time to decode with a state of the art neural translation system to just over 140 msec per sentence on a single CPU core for English-German.