Automatic Fashion Knowledge Extraction from Social Media
This addresses the need for automated fashion insights for users, but appears incremental as it builds on existing multimodal and noise-handling techniques.
The paper tackles the problem of automatically extracting fashion knowledge from social media by unifying occasion, person, and clothing discovery across images, texts, and metadata, resulting in a system demonstrated through a website.
Fashion knowledge plays a pivotal role in helping people in their dressing. In this paper, we present a novel system to automatically harvest fashion knowledge from social media. It unifies three tasks of occasion, person and clothing discovery from multiple modalities of images, texts and metadata. A contextualized fashion concept learning model is applied to leverage the rich contextual information for improving the fashion concept learning performance. At the same time, to counter the label noise within training data, we employ a weak label modeling method to further boost the performance. We build a website to demonstrate the quality of fashion knowledge extracted by our system.