Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer
This work addresses the problem of resource-intensive fine-tuning for text style transfer, offering a more efficient solution for NLP practitioners, though it is incremental as it builds on existing adapter methods.
The paper tackles the challenge of adapting large language models for multiple-attribute text style transfer by introducing Adapter-TST, a parameter-efficient framework that freezes pre-trained parameters and uses neural adapters to capture attributes like sentiment and tense, achieving state-of-the-art performance with significantly fewer computational resources.
Adapting a large language model for multiple-attribute text style transfer via fine-tuning can be challenging due to the significant amount of computational resources and labeled data required for the specific task. In this paper, we address this challenge by introducing AdapterTST, a framework that freezes the pre-trained model's original parameters and enables the development of a multiple-attribute text style transfer model. Using BART as the backbone model, Adapter-TST utilizes different neural adapters to capture different attribute information, like a plug-in connected to BART. Our method allows control over multiple attributes, like sentiment, tense, voice, etc., and configures the adapters' architecture to generate multiple outputs respected to attributes or compositional editing on the same sentence. We evaluate the proposed model on both traditional sentiment transfer and multiple-attribute transfer tasks. The experiment results demonstrate that Adapter-TST outperforms all the state-of-the-art baselines with significantly lesser computational resources. We have also empirically shown that each adapter is able to capture specific stylistic attributes effectively and can be configured to perform compositional editing.