Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization
This addresses content preservation in text style transfer, an incremental improvement for natural language processing applications.
The paper tackled the problem of content loss in text style transfer by proposing a method that uses reverse attention to implicitly remove style information and conditional layer normalization to create content-dependent style representations, resulting in outperforming state-of-the-art baselines by a large margin in content preservation.
Text style transfer aims to alter the style (e.g., sentiment) of a sentence while preserving its content. A common approach is to map a given sentence to content representation that is free of style, and the content representation is fed to a decoder with a target style. Previous methods in filtering style completely remove tokens with style at the token level, which incurs the loss of content information. In this paper, we propose to enhance content preservation by implicitly removing the style information of each token with reverse attention, and thereby retain the content. Furthermore, we fuse content information when building the target style representation, making it dynamic with respect to the content. Our method creates not only style-independent content representation, but also content-dependent style representation in transferring style. Empirical results show that our method outperforms the state-of-the-art baselines by a large margin in terms of content preservation. In addition, it is also competitive in terms of style transfer accuracy and fluency.