Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generation
This addresses the challenge of identity preservation in video generation for users of generative AI tools, offering a more efficient and compatible solution, though it is incremental as it builds on pre-trained models.
The paper tackles the problem of generating high-fidelity human videos that match user-specified identities by proposing Stand-In, a lightweight and plug-and-play framework that achieves excellent results in video quality and identity preservation with only ~1% additional parameters and 2000 training pairs, outperforming full-parameter training methods.
Generating high-fidelity human videos that match user-specified identities is important yet challenging in the field of generative AI. Existing methods often rely on an excessive number of training parameters and lack compatibility with other AIGC tools. In this paper, we propose Stand-In, a lightweight and plug-and-play framework for identity preservation in video generation. Specifically, we introduce a conditional image branch into the pre-trained video generation model. Identity control is achieved through restricted self-attentions with conditional position mapping, and can be learned quickly with only 2000 pairs. Despite incorporating and training just $\sim$1% additional parameters, our framework achieves excellent results in video quality and identity preservation, outperforming other full-parameter training methods. Moreover, our framework can be seamlessly integrated for other tasks, such as subject-driven video generation, pose-referenced video generation, stylization, and face swapping.