Expanding the Content-Style Frontier: a Balanced Subspace Blending Approach for Content-Style LoRA Fusion
This work addresses a specific bottleneck in personalization and stylization for text-to-image generation, offering an incremental improvement over prior techniques.
The paper tackles the problem of content loss at high style intensities in text-to-image diffusion models by proposing a balanced subspace blending approach, which improves content similarity across varying intensities and achieves lower Inverted Generational Distance and Generational Distance scores compared to existing methods.
Recent advancements in text-to-image diffusion models have significantly improved the personalization and stylization of generated images. However, previous studies have only assessed content similarity under a single style intensity. In our experiments, we observe that increasing style intensity leads to a significant loss of content features, resulting in a suboptimal content-style frontier. To address this, we propose a novel approach to expand the content-style frontier by leveraging Content-Style Subspace Blending and a Content-Style Balance loss. Our method improves content similarity across varying style intensities, significantly broadening the content-style frontier. Extensive experiments demonstrate that our approach outperforms existing techniques in both qualitative and quantitative evaluations, achieving superior content-style trade-off with significantly lower Inverted Generational Distance (IGD) and Generational Distance (GD) scores compared to current methods.