Significance of Skeleton-based Features in Virtual Try-On
This addresses a specific issue in e-retailing for users trying on clothes virtually, but it is incremental as it builds on existing VTON methods.
The paper tackled the problem of inconsistent results in Virtual Try-On (VTON) for poses with bent or crossed arms, especially for long-sleeved outfits, by proposing a method that segments clothing into parts and uses hand-crafted geometric features and learning-based modules, achieving effectiveness compared to benchmark methods.
The idea of \textit{Virtual Try-ON} (VTON) benefits e-retailing by giving an user the convenience of trying a clothing at the comfort of their home. In general, most of the existing VTON methods produce inconsistent results when a person posing with his arms folded i.e., bent or crossed, wants to try an outfit. The problem becomes severe in the case of long-sleeved outfits. As then, for crossed arm postures, overlap among different clothing parts might happen. The existing approaches, especially the warping-based methods employing \textit{Thin Plate Spline (TPS)} transform can not tackle such cases. To this end, we attempt a solution approach where the clothing from the source person is segmented into semantically meaningful parts and each part is warped independently to the shape of the person. To address the bending issue, we employ hand-crafted geometric features consistent with human body geometry for warping the source outfit. In addition, we propose two learning-based modules: a synthesizer network and a mask prediction network. All these together attempt to produce a photo-realistic, pose-robust VTON solution without requiring any paired training data. Comparison with some of the benchmark methods clearly establishes the effectiveness of the approach.