FastFace: Tuning Identity Preservation in Distilled Diffusion via Guidance and Attention
This work addresses the problem of slow inference in personalized image generation for users of diffusion models, though it appears incremental as it builds on existing adapters and distillation techniques.
The paper tackles the challenge of adapting identity-preserving adapters to distilled diffusion models without retraining, proposing the FastFace framework that redesigns classifier-free guidance and attention mechanisms to improve identity similarity and fidelity in few-step generation.
In latest years plethora of identity-preserving adapters for a personalized generation with diffusion models have been released. Their main disadvantage is that they are dominantly trained jointly with base diffusion models, which suffer from slow multi-step inference. This work aims to tackle the challenge of training-free adaptation of pretrained ID-adapters to diffusion models accelerated via distillation - through careful re-design of classifier-free guidance for few-step stylistic generation and attention manipulation mechanisms in decoupled blocks to improve identity similarity and fidelity, we propose universal FastFace framework. Additionally, we develop a disentangled public evaluation protocol for id-preserving adapters.