TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings
This addresses the need for efficient and effective style transfer for NLP applications, though it is incremental as it builds on existing authorship embedding and few-shot techniques.
The authors tackled the problem of few-shot text style transfer by introducing TinyStyler, a lightweight method using a small language model and authorship embeddings, which outperformed GPT-4 on authorship style transfer and recent controllable text generation methods on formal-informal transfer.
The goal of text style transfer is to transform the style of texts while preserving their original meaning, often with only a few examples of the target style. Existing style transfer methods generally rely on the few-shot capabilities of large language models or on complex controllable text generation approaches that are inefficient and underperform on fluency metrics. We introduce TinyStyler, a lightweight but effective approach, which leverages a small language model (800M params) and pre-trained authorship embeddings to perform efficient, few-shot text style transfer. We evaluate on the challenging task of authorship style transfer and find TinyStyler outperforms strong approaches such as GPT-4. We also evaluate TinyStyler's ability to perform text attribute style transfer (formal $\leftrightarrow$ informal) with automatic and human evaluations and find that the approach outperforms recent controllable text generation methods. Our model has been made publicly available at https://huggingface.co/tinystyler/tinystyler .