NSL-MT: Linguistically Informed Negative Samples for Efficient Machine Translation in Low-Resource Languages
This addresses data scarcity in machine translation for low-resource languages, offering a data-efficient training method, though it is incremental as it builds on existing training paradigms.
The paper tackles the problem of machine translation in low-resource languages by introducing NSL-MT, a training method that uses linguistically informed negative samples to penalize invalid outputs, resulting in BLEU gains of 3-12% for well-performing models and 56-89% for poorly performing ones, along with a 5x data efficiency improvement.
We introduce Negative Space Learning MT (NSL-MT), a training method that teaches models what not to generate by encoding linguistic constraints as severity-weighted penalties in the loss function. NSL-MT increases limited parallel data with synthetically generated violations of target language grammar, explicitly penalizing the model when it assigns high probability to these linguistically invalid outputs. We demonstrate that NSL-MT delivers improvements across all architectures: 3-12\% BLEU gains for well-performing models and 56-89\% gains for models lacking descent initial support. Furthermore, NSL-MT provides a 5x data efficiency multiplier -- training with 1,000 examples matches or exceeds normal training with 5,000 examples. Thus, NSL-MT provides a data-efficient alternative training method for settings where there is limited annotated parallel corporas.