Concat-ID: Towards Universal Identity-Preserving Video Synthesis
This provides a scalable solution for applications like virtual try-on and background-controllable generation, though it appears incremental in improving identity preservation in video synthesis.
The paper tackles the problem of identity-preserving video generation by proposing Concat-ID, a framework that uses variational autoencoders and 3D self-attention to maintain identity consistency while enabling facial editability, achieving superior performance over existing methods in single and multi-identity scenarios.
We present Concat-ID, a unified framework for identity-preserving video generation. Concat-ID employs variational autoencoders to extract image features, which are then concatenated with video latents along the sequence dimension. It relies exclusively on inherent 3D self-attention mechanisms to incorporate them, eliminating the need for additional parameters or modules. A novel cross-video pairing strategy and a multi-stage training regimen are introduced to balance identity consistency and facial editability while enhancing video naturalness. Extensive experiments demonstrate Concat-ID's superiority over existing methods in both single and multi-identity generation, as well as its seamless scalability to multi-subject scenarios, including virtual try-on and background-controllable generation. Concat-ID establishes a new benchmark for identity-preserving video synthesis, providing a versatile and scalable solution for a wide range of applications.