Shape-aware Text-driven Layered Video Editing
This addresses a limitation in video editing for applications requiring object shape modifications, though it appears incremental by building on existing layered representation approaches.
The paper tackles the problem of editing object shapes in videos while maintaining temporal consistency, achieving shape-aware consistent video editing that compares favorably with state-of-the-art methods.
Temporal consistency is essential for video editing applications. Existing work on layered representation of videos allows propagating edits consistently to each frame. These methods, however, can only edit object appearance rather than object shape changes due to the limitation of using a fixed UV mapping field for texture atlas. We present a shape-aware, text-driven video editing method to tackle this challenge. To handle shape changes in video editing, we first propagate the deformation field between the input and edited keyframe to all frames. We then leverage a pre-trained text-conditioned diffusion model as guidance for refining shape distortion and completing unseen regions. The experimental results demonstrate that our method can achieve shape-aware consistent video editing and compare favorably with the state-of-the-art.