Bilingual is At Least Monolingual (BALM): A Novel Translation Algorithm that Encodes Monolingual Priors
This addresses the problem of inefficient translation models for researchers and practitioners by proposing a novel framework, though it appears incremental as it builds on existing embedding methods.
The paper tackles machine translation by incorporating monolingual priors into the pipeline, achieving near-state-of-the-art BLEU scores on English-to-German translation with a simple feedforward network.
State-of-the-art machine translation (MT) models do not use knowledge of any single language's structure; this is the equivalent of asking someone to translate from English to German while knowing neither language. BALM is a framework incorporates monolingual priors into an MT pipeline; by casting input and output languages into embedded space using BERT, we can solve machine translation with much simpler models. We find that English-to-German translation on the Multi30k dataset can be solved with a simple feedforward network under the BALM framework with near-SOTA BLEU scores.