From Keypoints to Realism: A Realistic and Accurate Virtual Try-on Network from 2D Images
This work solves the challenge of accurate and realistic virtual try-on for e-commerce and fashion applications, though it appears incremental by building on existing keypoint-based approaches.
The paper tackles the problem of generating realistic virtual try-on images from 2D photos by addressing failures in reproducing garment details and generalizing to new scenarios, resulting in a method that preserves garment shape and texture with high visual quality.
The aim of image-based virtual try-on is to generate realistic images of individuals wearing target garments, ensuring that the pose, body shape and characteristics of the target garment are accurately preserved. Existing methods often fail to reproduce the fine details of target garments effectively and lack generalizability to new scenarios. In the proposed method, the person's initial garment is completely removed. Subsequently, a precise warping is performed using the predicted keypoints to fully align the target garment with the body structure and pose of the individual. Based on the warped garment, a body segmentation map is more accurately predicted. Then, using an alignment-aware segment normalization, the misaligned areas between the warped garment and the predicted garment region in the segmentation map are removed. Finally, the generator produces the final image with high visual quality, reconstructing the precise characteristics of the target garment, including its overall shape and texture. This approach emphasizes preserving garment characteristics and improving adaptability to various poses, providing better generalization for diverse applications.