INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation
This work addresses efficiency and performance issues in machine translation for NLP practitioners, offering an incremental improvement over existing kNN-MT methods.
The paper tackles the inference overhead and non-smooth representation space in kNN-Machine Translation by proposing INK, a training framework that adjusts neighbor representations with few parameters, achieving average gains of 1.99 COMET and 1.0 BLEU with 0.02x memory and 1.9x speedup.
Neural machine translation has achieved promising results on many translation tasks. However, previous studies have shown that neural models induce a non-smooth representation space, which harms its generalization results. Recently, kNN-MT has provided an effective paradigm to smooth the prediction based on neighbor representations during inference. Despite promising results, kNN-MT usually requires large inference overhead. We propose an effective training framework INK to directly smooth the representation space via adjusting representations of kNN neighbors with a small number of new parameters. The new parameters are then used to refresh the whole representation datastore to get new kNN knowledge asynchronously. This loop keeps running until convergence. Experiments on four benchmark datasets show that \method achieves average gains of 1.99 COMET and 1.0 BLEU, outperforming the state-of-the-art kNN-MT system with 0.02x memory space and 1.9x inference speedup.