CEV-LM: Controlled Edit Vector Language Model for Shaping Natural Language Generations
This addresses the need for tailored text generations in applications like audience-specific content, but it is incremental as it builds on existing control approaches.
The paper tackled the problem of controlling text shape metrics like pacing in language model generations, and introduced CEV-LM, which achieved significantly more targeted and precise control over speed, volume, and circuitousness while preserving semantics and using fewer parameters.
As large-scale language models become the standard for text generation, there is a greater need to tailor the generations to be more or less concise, targeted, and informative, depending on the audience/application. Existing control approaches primarily adjust the semantic (e.g., emotion, topics), structural (e.g., syntax tree, parts-of-speech), and lexical (e.g., keyword/phrase inclusion) properties of text, but are insufficient to accomplish complex objectives such as pacing which control the complexity and readability of the text. In this paper, we introduce CEV-LM - a lightweight, semi-autoregressive language model that utilizes constrained edit vectors to control three complementary metrics (speed, volume, and circuitousness) that quantify the shape of text (e.g., pacing of content). We study an extensive set of state-of-the-art CTG models and find that CEV-LM provides significantly more targeted and precise control of these three metrics while preserving semantic content, using less training data, and containing fewer parameters.