Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastings
This work addresses a foundational issue in natural language processing by providing a method to sample from MLMs as energy-based models, which is incremental as it builds on existing MLM frameworks.
The paper tackled the problem of whether masked language models (MLMs) specify a principled probability distribution over sequences by interpreting them as energy-based models and proposing two energy parametrizations, with results showing higher quality samples than other undirected generation approaches in tasks like unconditional generation and machine translation.
While recent work has shown that scores from models trained by the ubiquitous masked language modeling (MLM) objective effectively discriminate probable from improbable sequences, it is still an open question if these MLMs specify a principled probability distribution over the space of possible sequences. In this paper, we interpret MLMs as energy-based sequence models and propose two energy parametrizations derivable from the trained MLMs. In order to draw samples correctly from these models, we develop a tractable sampling scheme based on the Metropolis--Hastings Monte Carlo algorithm. In our approach, samples are proposed from the same masked conditionals used for training the masked language models, and they are accepted or rejected based on their energy values according to the target distribution. We validate the effectiveness of the proposed parametrizations by exploring the quality of samples drawn from these energy-based models for both open-ended unconditional generation and a conditional generation task of machine translation. We theoretically and empirically justify our sampling algorithm by showing that the masked conditionals on their own do not yield a Markov chain whose stationary distribution is that of our target distribution, and our approach generates higher quality samples than other recently proposed undirected generation approaches (Wang et al., 2019, Ghazvininejad et al., 2019).