Non-Parallel Text Style Transfer with Self-Parallel Supervision
This work addresses a key bottleneck in text style transfer for natural language processing applications, offering improved performance on tasks like sentiment and political stance transfer, though it is incremental as it builds on existing language models.
The paper tackled the problem of text style transfer with non-parallel datasets, where models often fail due to weak supervision, and proposed LaMer, a framework that mines roughly parallel expressions using scene graphs and employs MLE training with imitation learning, achieving qualitative advances in transfer accuracy, content preservation, and fluency on benchmark tasks.
The performance of existing text style transfer models is severely limited by the non-parallel datasets on which the models are trained. In non-parallel datasets, no direct mapping exists between sentences of the source and target style; the style transfer models thus only receive weak supervision of the target sentences during training, which often leads the model to discard too much style-independent information, or utterly fail to transfer the style. In this work, we propose LaMer, a novel text style transfer framework based on large-scale language models. LaMer first mines the roughly parallel expressions in the non-parallel datasets with scene graphs, and then employs MLE training, followed by imitation learning refinement, to leverage the intrinsic parallelism within the data. On two benchmark tasks (sentiment & formality transfer) and a newly proposed challenging task (political stance transfer), our model achieves qualitative advances in transfer accuracy, content preservation, and fluency. Further empirical and human evaluations demonstrate that our model not only makes training more efficient, but also generates more readable and diverse expressions than previous models.