P-Masking: Power Law Masking Improves Multi-attribute Controlled Generation
This addresses the need for precise and adaptable control over multiple attributes in text generation, representing a novel method for a known bottleneck rather than a foundational advancement.
The paper tackles the problem of controlled text generation with multiple linguistic attributes by introducing LingGen with a dynamic P-MASKING strategy, achieving superior performance in attribute control accuracy and text fluency compared to state-of-the-art models.
We introduce LingGen, a novel approach for controlled text generation that offers precise control over a wide array of linguistic attributes, even as the number of attributes varies. LingGen employs a dynamic P-MASKING strategy, which samples masking rates from a power law distribution during training. This innovative approach enables the model to develop robust representations and adapt its attribute control capabilities across a variable number of attributes, from a single attribute to multiple complex configurations. The P-MASKING technique enhances LingGen's ability to manage different levels of attribute visibility, resulting in superior performance in multi-attribute generation tasks. Our experiments demonstrate that LingGen surpasses current state-of-the-art models in both attribute control accuracy and text fluency, particularly excelling in scenarios with varying attribute demands. Additionally, our ablation studies highlight the effectiveness of P-MASKING and the influence of different base language models on performance. These findings demonstrate LingGen's potential for applications requiring precise and adaptable control over multiple linguistic attributes in text generation.