Partial Visual-Semantic Embedding: Fashion Intelligence System with Sensitive Part-by-Part Learning
This addresses a domain-specific problem for fashion intelligence systems by enabling part-sensitive learning, though it is incremental as it builds on existing VSE models.
The paper tackles the problem of quantifying abstract fashion expressions like 'casual' using visual-semantic embedding, but existing models fail with multi-part fashion images. The proposed partial VSE model enables sensitive part-by-part learning, achieving superior performance in tasks like image retrieval and reordering without increased computational complexity, as shown in evaluations.
In this study, we propose a technology called the Fashion Intelligence System based on the visual-semantic embedding (VSE) model to quantify abstract and complex expressions unique to fashion, such as ''casual,'' ''adult-casual,'' and ''office-casual,'' and to support users' understanding of fashion. However, the existing VSE model does not support the situations in which the image is composed of multiple parts such as hair, tops, pants, skirts, and shoes. We propose partial VSE, which enables sensitive learning for each part of the fashion coordinates. The proposed model partially learns embedded representations. This helps retain the various existing practical functionalities and enables image-retrieval tasks in which changes are made only to the specified parts and image reordering tasks that focus on the specified parts. This was not possible with conventional models. Based on both the qualitative and quantitative evaluation experiments, we show that the proposed model is superior to conventional models without increasing the computational complexity.