Exploiting Social Media Content for Self-Supervised Style Transfer
This work addresses style transfer tasks like civil rephrasing and formality for natural language processing applications, representing an incremental improvement by integrating existing methods.
The paper tackled style transfer by proposing a Self-Supervised Style Transfer (3ST) model that combines self-supervised neural machine translation methods with unsupervised techniques to leverage social media data, achieving the best balance in fluency, content preservation, and attribute transfer accuracy and outperforming state-of-the-art models in automatic and human evaluations.
Recent research on style transfer takes inspiration from unsupervised neural machine translation (UNMT), learning from large amounts of non-parallel data by exploiting cycle consistency loss, back-translation, and denoising autoencoders. By contrast, the use of self-supervised NMT (SSNMT), which leverages (near) parallel instances hidden in non-parallel data more efficiently than UNMT, has not yet been explored for style transfer. In this paper we present a novel Self-Supervised Style Transfer (3ST) model, which augments SSNMT with UNMT methods in order to identify and efficiently exploit supervisory signals in non-parallel social media posts. We compare 3ST with state-of-the-art (SOTA) style transfer models across civil rephrasing, formality and polarity tasks. We show that 3ST is able to balance the three major objectives (fluency, content preservation, attribute transfer accuracy) the best, outperforming SOTA models on averaged performance across their tested tasks in automatic and human evaluation.