Neutral Editing Framework for Diffusion-based Video Editing
This addresses a significant limitation in video editing for AI and creative applications, as existing methods are limited to rigid edits like style transfer.
The paper tackles the challenge of complex non-rigid video editing, such as changing the motion of objects, by proposing the Neutral Editing (NeuEdit) framework, which enables this without auxiliary aids like masks or captions.
Text-conditioned image editing has succeeded in various types of editing based on a diffusion framework. Unfortunately, this success did not carry over to a video, which continues to be challenging. Existing video editing systems are still limited to rigid-type editing such as style transfer and object overlay. To this end, this paper proposes Neutral Editing (NeuEdit) framework to enable complex non-rigid editing by changing the motion of a person/object in a video, which has never been attempted before. NeuEdit introduces a concept of `neutralization' that enhances a tuning-editing process of diffusion-based editing systems in a model-agnostic manner by leveraging input video and text without any other auxiliary aids (e.g., visual masks, video captions). Extensive experiments on numerous videos demonstrate adaptability and effectiveness of the NeuEdit framework. The website of our work is available here: https://neuedit.github.io