Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network
This work addresses the need for explainable AI in fashion recommendation, offering a diagnostic tool for outfit compatibility, though it is incremental by building on existing compatibility prediction methods.
The paper tackles the problem of predicting and explaining outfit compatibility by introducing a multi-layered comparison network that learns from pairwise similarities and uses gradients for diagnosis, achieving advantages in prediction performance and diagnosis ability over prior state-of-the-art methods.
Existing works about fashion outfit compatibility focus on predicting the overall compatibility of a set of fashion items with their information from different modalities. However, there are few works explore how to explain the prediction, which limits the persuasiveness and effectiveness of the model. In this work, we propose an approach to not only predict but also diagnose the outfit compatibility. We introduce an end-to-end framework for this goal, which features for: (1) The overall compatibility is learned from all type-specified pairwise similarities between items, and the backpropagation gradients are used to diagnose the incompatible factors. (2) We leverage the hierarchy of CNN and compare the features at different layers to take into account the compatibilities of different aspects from the low level (such as color, texture) to the high level (such as style). To support the proposed method, we build a new type-specified outfit dataset named Polyvore-T based on Polyvore dataset. We compare our method with the prior state-of-the-art in two tasks: outfit compatibility prediction and fill-in-the-blank. Experiments show that our approach has advantages in both prediction performance and diagnosis ability.