Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation
This addresses concerns about the suitability of reinforcement learning for neural machine translation, offering insights for researchers in NLP and machine translation.
The paper revisits criticisms of policy gradient algorithms for neural machine translation, finding that exploration and reward scaling are important and providing empirical counter-evidence to claims that their success is due to output distribution shape rather than reward.
Policy gradient algorithms have found wide adoption in NLP, but have recently become subject to criticism, doubting their suitability for NMT. Choshen et al. (2020) identify multiple weaknesses and suspect that their success is determined by the shape of output distributions rather than the reward. In this paper, we revisit these claims and study them under a wider range of configurations. Our experiments on in-domain and cross-domain adaptation reveal the importance of exploration and reward scaling, and provide empirical counter-evidence to these claims.