Inference Strategies for Machine Translation with Conditional Masking
This work addresses a specific inference bottleneck for researchers and practitioners using CMLMs in translation tasks, representing an incremental improvement.
The paper tackles the problem of determining optimal inference strategies for conditional masked language models (CMLMs) in machine translation, showing that a proposed thresholding strategy outperforms the standard mask-predict algorithm with improved performance.
Conditional masked language model (CMLM) training has proven successful for non-autoregressive and semi-autoregressive sequence generation tasks, such as machine translation. Given a trained CMLM, however, it is not clear what the best inference strategy is. We formulate masked inference as a factorization of conditional probabilities of partial sequences, show that this does not harm performance, and investigate a number of simple heuristics motivated by this perspective. We identify a thresholding strategy that has advantages over the standard "mask-predict" algorithm, and provide analyses of its behavior on machine translation tasks.