Faster Re-translation Using Non-Autoregressive Model For Simultaneous Neural Machine Translation
This work provides a faster re-translation system for applications like live event subtitling and real-time video-call translation, which is particularly beneficial for resource-constrained devices.
The paper addresses the problem of slow re-translation in simultaneous neural machine translation, proposing a non-autoregressive model (FReTNA) to replace existing autoregressive models (ReTA). FReTNA reduces inference times by a factor of 20 compared to ReTA with a small drop in quality, and outperforms streaming Wait-k models by being 1.5 times faster while also improving translation quality.
Recently, simultaneous translation has gathered a lot of attention since it enables compelling applications such as subtitle translation for a live event or real-time video-call translation. Some of these translation applications allow editing of partial translation giving rise to re-translation approaches. The current re-translation approaches are based on autoregressive sequence generation models (ReTA), which generate tar-get tokens in the (partial) translation sequentially. The multiple re-translations with sequential generation inReTAmodelslead to an increased inference time gap between the incoming source input and the corresponding target output as the source input grows. Besides, due to the large number of inference operations involved, the ReTA models are not favorable for resource-constrained devices. In this work, we propose a faster re-translation system based on a non-autoregressive sequence generation model (FReTNA) to overcome the aforementioned limitations. We evaluate the proposed model on multiple translation tasks and our model reduces the inference times by several orders and achieves a competitive BLEUscore compared to the ReTA and streaming (Wait-k) models.The proposed model reduces the average computation time by a factor of 20 when compared to the ReTA model by incurring a small drop in the translation quality. It also outperforms the streaming-based Wait-k model both in terms of computation time (1.5 times lower) and translation quality.