IMTalker: Efficient Audio-driven Talking Face Generation with Implicit Motion Transfer
This addresses the challenge of efficient and high-fidelity talking face generation for applications like virtual avatars and video editing, though it is incremental as it builds on prior motion transfer techniques.
The paper tackles the problem of generating realistic talking faces from audio, where existing methods struggle with complex motions and identity drift, by proposing IMTalker, which achieves state-of-the-art quality with improved motion accuracy and identity preservation, operating at 40-42 FPS.
Talking face generation aims to synthesize realistic speaking portraits from a single image, yet existing methods often rely on explicit optical flow and local warping, which fail to model complex global motions and cause identity drift. We present IMTalker, a novel framework that achieves efficient and high-fidelity talking face generation through implicit motion transfer. The core idea is to replace traditional flow-based warping with a cross-attention mechanism that implicitly models motion discrepancy and identity alignment within a unified latent space, enabling robust global motion rendering. To further preserve speaker identity during cross-identity reenactment, we introduce an identity-adaptive module that projects motion latents into personalized spaces, ensuring clear disentanglement between motion and identity. In addition, a lightweight flow-matching motion generator produces vivid and controllable implicit motion vectors from audio, pose, and gaze cues. Extensive experiments demonstrate that IMTalker surpasses prior methods in motion accuracy, identity preservation, and audio-lip synchronization, achieving state-of-the-art quality with superior efficiency, operating at 40 FPS for video-driven and 42 FPS for audio-driven generation on an RTX 4090 GPU. We will release our code and pre-trained models to facilitate applications and future research.