Distilling Text Style Transfer With Self-Explanation From LLMs
This work addresses the challenge of text style transfer for natural language processing applications, offering an incremental improvement by leveraging LLMs for better efficiency and transparency.
The authors tackled the problem of text style transfer with limited parallel data by proposing CoTeX, a framework that distills large language models' reasoning into more efficient models, achieving superior performance over existing methods across four datasets, especially in low-resource settings.
Text Style Transfer (TST) seeks to alter the style of text while retaining its core content. Given the constraints of limited parallel datasets for TST, we propose CoTeX, a framework that leverages large language models (LLMs) alongside chain-of-thought (CoT) prompting to facilitate TST. CoTeX distills the complex rewriting and reasoning capabilities of LLMs into more streamlined models capable of working with both non-parallel and parallel data. Through experimentation across four TST datasets, CoTeX is shown to surpass traditional supervised fine-tuning and knowledge distillation methods, particularly in low-resource settings. We conduct a comprehensive evaluation, comparing CoTeX against current unsupervised, supervised, in-context learning (ICL) techniques, and instruction-tuned LLMs. Furthermore, CoTeX distinguishes itself by offering transparent explanations for its style transfer process.