Multi-Task Curriculum Transfer Deep Learning of Clothing Attributes
This work addresses the problem of limited training data for fine-grained clothing recognition in real-world settings, which is incremental as it builds on existing transfer learning and multi-task methods.
The paper tackles the challenge of recognizing fine-grained clothing attributes in unconstrained images by developing a Multi-Task Curriculum Transfer deep learning method that transfers knowledge from well-controlled shop images to in-the-wild street images, achieving state-of-the-art results on the X-Domain benchmark and showing notable advantages with small training data.
Recognising detailed clothing characteristics (fine-grained attributes) in unconstrained images of people in-the-wild is a challenging task for computer vision, especially when there is only limited training data from the wild whilst most data available for model learning are captured in well-controlled environments using fashion models (well lit, no background clutter, frontal view, high-resolution). In this work, we develop a deep learning framework capable of model transfer learning from well-controlled shop clothing images collected from web retailers to in-the-wild images from the street. Specifically, we formulate a novel Multi-Task Curriculum Transfer (MTCT) deep learning method to explore multiple sources of different types of web annotations with multi-labelled fine-grained attributes. Our multi-task loss function is designed to extract more discriminative representations in training by jointly learning all attributes, and our curriculum strategy exploits the staged easy-to-complex transfer learning motivated by cognitive studies. We demonstrate the advantages of the MTCT model over the state-of-the-art methods on the X-Domain benchmark, a large scale clothing attribute dataset. Moreover, we show that the MTCT model has a notable advantage over contemporary models when the training data size is small.