Focused Prefix Tuning for Controllable Text Generation
This addresses performance degradation in controllable text generation for NLP applications, but it is incremental as it builds on existing prefix tuning methods.
The paper tackled the problem of unannotated attributes degrading performance in controllable text generation by proposing focused prefix tuning (FPT), which achieved better control accuracy and fluency in single-attribute tasks and comparable accuracy with flexibility in multi-attribute tasks.
In a controllable text generation dataset, there exist unannotated attributes that could provide irrelevant learning signals to models that use it for training and thus degrade their performance. We propose focused prefix tuning(FPT) to mitigate the problem and to enable the control to focus on the desired attribute. Experimental results show that FPT can achieve better control accuracy and text fluency than baseline models in single-attribute control tasks. In multi-attribute control tasks, FPT achieves comparable control accuracy with the state-of-the-art approach while keeping the flexibility to control new attributes without retraining existing models.