Conditioned Natural Language Generation using only Unconditioned Language Model: An Exploration
This addresses the challenge of conditioned natural language generation for users needing tailored text outputs, presenting an incremental improvement by simplifying existing methods.
The paper tackled the problem of generating text conditioned on user inputs like topics or attributes without modifying the original language model architecture, and found that using only an unconditioned language model is sufficient, achieving competitive results in fluency and diversity as evaluated automatically and by humans.
Transformer-based language models have shown to be very powerful for natural language generation (NLG). However, text generation conditioned on some user inputs, such as topics or attributes, is non-trivial. Past approach relies on either modifying the original LM architecture, re-training the LM on corpora with attribute labels, or having separately trained `guidance models' to guide text generation in decoding. We argued that the above approaches are not necessary, and the original unconditioned LM is sufficient for conditioned NLG. We evaluated our approaches by the samples' fluency and diversity with automated and human evaluation.