Vanast: Virtual Try-On with Human Image Animation via Synthetic Triplet Supervision
This work solves the problem of coherent virtual try-on and animation for applications in fashion and entertainment, though it appears incremental as it builds on existing video diffusion and data generation techniques.
The paper tackles the problem of generating garment-transferred human animation videos from a single image, garment images, and pose guidance, addressing issues like identity drift and distortion in conventional two-stage pipelines by performing the process in a unified step, resulting in high-fidelity, identity-consistent animations.
We present Vanast, a unified framework that generates garment-transferred human animation videos directly from a single human image, garment images, and a pose guidance video. Conventional two-stage pipelines treat image-based virtual try-on and pose-driven animation as separate processes, which often results in identity drift, garment distortion, and front-back inconsistency. Our model addresses these issues by performing the entire process in a single unified step to achieve coherent synthesis. To enable this setting, we construct large-scale triplet supervision. Our data generation pipeline includes generating identity-preserving human images in alternative outfits that differ from garment catalog images, capturing full upper and lower garment triplets to overcome the single-garment-posed video pair limitation, and assembling diverse in-the-wild triplets without requiring garment catalog images. We further introduce a Dual Module architecture for video diffusion transformers to stabilize training, preserve pretrained generative quality, and improve garment accuracy, pose adherence, and identity preservation while supporting zero-shot garment interpolation. Together, these contributions allow Vanast to produce high-fidelity, identity-consistent animation across a wide range of garment types.