FLDM-VTON: Faithful Latent Diffusion Model for Virtual Try-on
This work addresses the issue of inaccurate clothing details in virtual try-on for fashion and e-commerce applications, representing an incremental improvement over existing latent diffusion models.
The paper tackled the problem of virtual try-on methods lacking faithfulness to clothing details like style and pattern, and proposed FLDM-VTON, which outperformed state-of-the-art baselines on VITON-HD and Dress Code datasets by generating photo-realistic images with improved faithfulness.
Despite their impressive generative performance, latent diffusion model-based virtual try-on (VTON) methods lack faithfulness to crucial details of the clothes, such as style, pattern, and text. To alleviate these issues caused by the diffusion stochastic nature and latent supervision, we propose a novel Faithful Latent Diffusion Model for VTON, termed FLDM-VTON. FLDM-VTON improves the conventional latent diffusion process in three major aspects. First, we propose incorporating warped clothes as both the starting point and local condition, supplying the model with faithful clothes priors. Second, we introduce a novel clothes flattening network to constrain generated try-on images, providing clothes-consistent faithful supervision. Third, we devise a clothes-posterior sampling for faithful inference, further enhancing the model performance over conventional clothes-agnostic Gaussian sampling. Extensive experimental results on the benchmark VITON-HD and Dress Code datasets demonstrate that our FLDM-VTON outperforms state-of-the-art baselines and is able to generate photo-realistic try-on images with faithful clothing details.