InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptation
This work addresses the data scarcity problem in instruction-based video editing by showing that a video generation backbone can be turned into a strong editor with limited data, benefiting researchers and practitioners in video editing.
InsEdit adapts a video generation model into an instruction-based video editor using only O(100)K video editing data, achieving state-of-the-art results among open-source methods on video instruction editing benchmarks.
Instruction-based video editing is a natural way to control video content with text, but adapting a video generation model into an editor usually appears data-hungry. At the same time, high-quality video editing data remains scarce. In this paper, we show that a video generation backbone can become a strong video editor without large scale video editing data. We present InsEdit, an instruction-based editing model built on HunyuanVideo-1.5. InsEdit combines a visual editing architecture with a video data pipeline based on Mutual Context Attention (MCA), which creates aligned video pairs where edits can begin in the middle of a clip rather than only from the first frame. With only O(100)K video editing data, InsEdit achieves state-of-the-art results among open-source methods on our video instruction editing benchmarks. In addition, because our training recipe also includes image editing data, the final model supports image editing without any modification.