Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation
It provides a new benchmark and metrics for evaluating temporal visual reasoning in VLMs, a previously underexplored area.
The paper introduces Animation2Code, a benchmark of 1,069 web animation videos with code, to evaluate whether VLMs can recover temporal dynamics in video-to-code generation. Results show current VLMs struggle with temporal consistency despite achieving high appearance similarity.
While recent vision-language models (VLMs) have achieved significant improvements on static visual-to-code tasks such as generating code for webpages, charts, or SVGs, it remains unclear whether they can recover temporal dynamics when motion is present. To this end, we introduce Animation2Code, a benchmark for evaluating temporal visual reasoning via reconstructing executable web animation code from videos. Animation2Code consists of 1,069 web animation videos with diverse visual appearances and motion patterns, paired with corresponding HTML/CSS/JavaScript implementations. We propose two human-aligned metrics, appearance similarity and temporal similarity, which allow us to disentangle visual fidelity from temporal alignment when comparing rendered animations against ground-truth samples. Benchmarking state-of-the-art VLMs on this dataset shows that current VLMs struggle to maintain temporal consistency in reconstruction, even when achieving high appearance similarity, including under finetuning and iterative refinement settings. Code and data are available at https://anya-ji.github.io/animation2code-website .