Regularizing Reasons for Outfit Evaluation with Gradient Penalty
This work addresses the need for interpretable AI in fashion recommendation, though it is incremental as it builds on existing methods like Grad-CAM and supervised learning with domain knowledge.
The paper tackles the problem of outfit evaluation by developing a system that provides judgments with explanations, using a new dataset (EVALUATION3) annotated with judgments, reasons, and attributes. The result is a network that achieves high precision and good interpretation, as shown in experimental results.
In this paper, we build an outfit evaluation system which provides feedbacks consisting of a judgment with a convincing explanation. The system is trained in a supervised manner which faithfully follows the domain knowledge in fashion. We create the EVALUATION3 dataset which is annotated with judgment, the decisive reason for the judgment, and all corresponding attributes (e.g. print, silhouette, and material \etc.). In the training process, features of all attributes in an outfit are first extracted and then concatenated as the input for the intra-factor compatibility net. Then, the inter-factor compatibility net is used to compute the loss for judgment. We penalize the gradient of judgment loss of so that our Grad-CAM-like reason is regularized to be consistent with the labeled reason. In inference, according to the obtained information of judgment, reason, and attributes, a user-friendly explanation sentence is generated by the pre-defined templates. The experimental results show that the obtained network combines the advantages of high precision and good interpretation.