Efficient Encoder-Free Pose Conditioning and Pose Control for Virtual Try-On
This work addresses pose control for online shopping customers using VTON, but it is incremental as it builds on a baseline model with minor modifications.
The paper tackled the challenge of pose control in Virtual Try-On (VTON) by incorporating pose data through spatial concatenation without adding parameters, finding that pose maps improved pose preservation and realism. It introduced a mixed-mask training strategy to enhance flexible product integration across varied poses.
As online shopping continues to grow, the demand for Virtual Try-On (VTON) technology has surged, allowing customers to visualize products on themselves by overlaying product images onto their own photos. An essential yet challenging condition for effective VTON is pose control, which ensures accurate alignment of products with the user's body while supporting diverse orientations for a more immersive experience. However, incorporating pose conditions into VTON models presents several challenges, including selecting the optimal pose representation, integrating poses without additional parameters, and balancing pose preservation with flexible pose control. In this work, we build upon a baseline VTON model that concatenates the reference image condition without external encoder, control network, or complex attention layers. We investigate methods to incorporate pose control into this pure concatenation paradigm by spatially concatenating pose data, comparing performance using pose maps and skeletons, without adding any additional parameters or module to the baseline model. Our experiments reveal that pose stitching with pose maps yields the best results, enhancing both pose preservation and output realism. Additionally, we introduce a mixed-mask training strategy using fine-grained and bounding box masks, allowing the model to support flexible product integration across varied poses and conditions.