StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples
This work addresses the challenge of separating style from content in text embeddings for NLP researchers and practitioners, representing an incremental improvement over existing methods.
The paper tackles the problem of content leakage in style representations by introducing StyleDistance, a method that uses synthetic paraphrases with controlled style variations to train content-independent style embeddings, resulting in improved performance on downstream applications.
Style representations aim to embed texts with similar writing styles closely and texts with different styles far apart, regardless of content. However, the contrastive triplets often used for training these representations may vary in both style and content, leading to potential content leakage in the representations. We introduce StyleDistance, a novel approach to training stronger content-independent style embeddings. We use a large language model to create a synthetic dataset of near-exact paraphrases with controlled style variations, and produce positive and negative examples across 40 distinct style features for precise contrastive learning. We assess the quality of our synthetic data and embeddings through human and automatic evaluations. StyleDistance enhances the content-independence of style embeddings, which generalize to real-world benchmarks and outperform leading style representations in downstream applications. Our model can be found at https://huggingface.co/StyleDistance/styledistance .