DVI: Disentangling Semantic and Visual Identity for Training-Free Personalized Generation
This addresses the issue of visual inconsistency in personalized image generation for users needing high-fidelity customization without training, though it is incremental as it builds on existing tuning-free methods.
The paper tackled the problem of 'Semantic-Visual Dissonance' in tuning-free identity customization, where facial fidelity clashed with visual context like lighting and texture, causing unnatural effects. The result was DVI, a zero-shot framework that disentangles identity into semantic and visual streams, using VAE latent statistics and parameter-free modulation to enhance visual consistency and atmospheric fidelity, outperforming state-of-the-art methods in IBench evaluations.
Recent tuning-free identity customization methods achieve high facial fidelity but often overlook visual context, such as lighting, skin texture, and environmental tone. This limitation leads to ``Semantic-Visual Dissonance,'' where accurate facial geometry clashes with the input's unique atmosphere, causing an unnatural ``sticker-like'' effect. We propose **DVI (Disentangled Visual-Identity)**, a zero-shot framework that orthogonally disentangles identity into fine-grained semantic and coarse-grained visual streams. Unlike methods relying solely on semantic vectors, DVI exploits the inherent statistical properties of the VAE latent space, utilizing mean and variance as lightweight descriptors for global visual atmosphere. We introduce a **Parameter-Free Feature Modulation** mechanism that adaptively modulates semantic embeddings with these visual statistics, effectively injecting the reference's ``visual soul'' without training. Furthermore, a **Dynamic Temporal Granularity Scheduler** aligns with the diffusion process, prioritizing visual atmosphere in early denoising stages while refining semantic details later. Extensive experiments demonstrate that DVI significantly enhances visual consistency and atmospheric fidelity without parameter fine-tuning, maintaining robust identity preservation and outperforming state-of-the-art methods in IBench evaluations.