BERTian Poetics: Constrained Composition with Masked LMs
This work addresses a niche problem for researchers in computational creativity and natural language processing, but it is incremental as it builds on existing methods for generation.
The paper tackled the problem of constrained text composition using masked language models as energy-based sequence models, demonstrating their application through a Metropolis-Hastings sampler and exploring the poetics of the OuLiPo movement.
Masked language models have recently been interpreted as energy-based sequence models that can be generated from using a Metropolis--Hastings sampler. This short paper demonstrates how this can be instrumentalized for constrained composition and explores the poetics implied by such a usage. Our focus on constraints makes it especially apt to understand the generated text through the poetics of the OuLiPo movement.