JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation
This work addresses the lack of datasets and benchmarks for joint audio-visual editing, enabling progress in this under-explored area.
The paper introduces JAVEdit-100k, the first large-scale dataset for joint audio-visual editing, and JAVEditBench, a benchmark for evaluation. The proposed JAVEdit model outperforms all baselines on five of six metrics.
While instruction-based video editing has seen significant progress, joint audio-visual editing remains constrained by the absence of dedicated datasets and benchmarks. To bridge this gap, we present JAVEdit-100k, the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing. Focusing on human-centric videos, JAVEdit-100k comprises approximately 100K editing triplets spanning five distinct categories, including subject editing and speech editing. This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism. Furthermore, to address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions across all editing categories. Finally, we propose JAVEdit, a pioneering baseline model for instruction-guided joint audio-visual editing. Experiments show that \model\ outperforms all baselines on five of six evaluation metrics.