FixTalk: Taming Identity Leakage for High-Quality Talking Head Generation in Extreme Cases
This addresses quality issues in talking head generation for applications like entertainment and communication, representing a novel method for a known bottleneck.
The paper tackles identity leakage and rendering artifacts in talking head generation by proposing FixTalk, a framework that decouples identity from motion features and uses leaked identity to fix artifacts, achieving superior performance compared to state-of-the-art methods.
Talking head generation is gaining significant importance across various domains, with a growing demand for high-quality rendering. However, existing methods often suffer from identity leakage (IL) and rendering artifacts (RA), particularly in extreme cases. Through an in-depth analysis of previous approaches, we identify two key insights: (1) IL arises from identity information embedded within motion features, and (2) this identity information can be leveraged to address RA. Building on these findings, this paper introduces FixTalk, a novel framework designed to simultaneously resolve both issues for high-quality talking head generation. Firstly, we propose an Enhanced Motion Indicator (EMI) to effectively decouple identity information from motion features, mitigating the impact of IL on generated talking heads. To address RA, we introduce an Enhanced Detail Indicator (EDI), which utilizes the leaked identity information to supplement missing details, thus fixing the artifacts. Extensive experiments demonstrate that FixTalk effectively mitigates IL and RA, achieving superior performance compared to state-of-the-art methods.