CosAvatar: Consistent and Animatable Portrait Video Tuning with Text Prompt
This addresses the challenge of inconsistent portrait editing for digital content creators, offering a user-friendly solution without requiring specific data prerequisites.
The paper tackles the problem of maintaining temporal and 3D consistency in text-guided portrait video editing by proposing CosAvatar, which uses a dynamic NeRF-based 3D representation to edit monocular videos based on text prompts, enabling animatable portraits with precise modifications.
Recently, text-guided digital portrait editing has attracted more and more attentions. However, existing methods still struggle to maintain consistency across time, expression, and view or require specific data prerequisites. To solve these challenging problems, we propose CosAvatar, a high-quality and user-friendly framework for portrait tuning. With only monocular video and text instructions as input, we can produce animatable portraits with both temporal and 3D consistency. Different from methods that directly edit in the 2D domain, we employ a dynamic NeRF-based 3D portrait representation to model both the head and torso. We alternate between editing the video frames' dataset and updating the underlying 3D portrait until the edited frames reach 3D consistency. Additionally, we integrate the semantic portrait priors to enhance the edited results, allowing precise modifications in specified semantic areas. Extensive results demonstrate that our proposed method can not only accurately edit portrait styles or local attributes based on text instructions but also support expressive animation driven by a source video.