Knowledge-Intensive Video Generation
For researchers in video generation, this work highlights the gap in factuality and practical usefulness of current models, establishing a new benchmark and evaluation metrics.
The paper introduces knowledge-intensive video generation (KIVI), where models generate videos from short information-seeking prompts, and constructs KIVI-Bench with 1,080 prompts for evaluation. Experiments on seven state-of-the-art models show they lag behind human performance, especially on visual properties and procedural operations.
Text-to-video generation has advanced rapidly in visual quality, but remains under-evaluated for factuality and practical usefulness. We introduce knowledge-intensive video generation (KIVI), where models generate videos from short information-seeking prompts that ask for explanations, procedures, or demonstrations. To evaluate this setting, we construct KIVI-Bench, a benchmark of 1,080 prompts, and propose automatic metrics for factuality and helpfulness. Human evaluation shows that our metrics significantly better align with human annotations than existing alternatives. Experiments on seven state-of-the-art video generation models show that current systems still lag behind human performance, especially on visual properties, procedural operations, and clear information presentation. These results highlight KIVI as a challenging direction for factual and instructionally useful video generation.