K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs
This addresses the need for efficient and effective LoRA fusion in diffusion models, offering a training-free solution that improves upon existing methods, though it is incremental in nature.
The paper tackles the problem of combining different LoRAs to generate images with learned subject and style without additional training, proposing K-LoRA, which outperforms state-of-the-art training-based methods in qualitative and quantitative results.
Recent studies have explored combining different LoRAs to jointly generate learned style and content. However, existing methods either fail to effectively preserve both the original subject and style simultaneously or require additional training. In this paper, we argue that the intrinsic properties of LoRA can effectively guide diffusion models in merging learned subject and style. Building on this insight, we propose K-LoRA, a simple yet effective training-free LoRA fusion approach. In each attention layer, K-LoRA compares the Top-K elements in each LoRA to be fused, determining which LoRA to select for optimal fusion. This selection mechanism ensures that the most representative features of both subject and style are retained during the fusion process, effectively balancing their contributions. Experimental results demonstrate that the proposed method effectively integrates the subject and style information learned by the original LoRAs, outperforming state-of-the-art training-based approaches in both qualitative and quantitative results.