Adversarial Decomposition of Text Representation
This addresses the need for more linguistically realistic and controllable text representations for NLP tasks, though it appears incremental as it builds on existing adversarial and decomposition methods.
The paper tackles the problem of decomposing text representations into independent vectors for specific aspects like social registers and language change, achieving fine-grained control and continuous style representation, and shows that the resulting embeddings significantly outperform regular autoencoder embeddings in paraphrase detection.
In this paper, we present a method for adversarial decomposition of text representation. This method can be used to decompose a representation of an input sentence into several independent vectors, each of them responsible for a specific aspect of the input sentence. We evaluate the proposed method on two case studies: the conversion between different social registers and diachronic language change. We show that the proposed method is capable of fine-grained controlled change of these aspects of the input sentence. It is also learning a continuous (rather than categorical) representation of the style of the sentence, which is more linguistically realistic. The model uses adversarial-motivational training and includes a special motivational loss, which acts opposite to the discriminator and encourages a better decomposition. Furthermore, we evaluate the obtained meaning embeddings on a downstream task of paraphrase detection and show that they significantly outperform the embeddings of a regular autoencoder.