Where to Look and How to Describe: Fashion Image Retrieval with an Attentional Heterogeneous Bilinear Network
This work addresses fashion product retrieval for e-commerce applications, but it is incremental as it builds on existing bilinear and attention methods.
The paper tackled fashion image retrieval by extracting appearance and localization information and their interactions, achieving satisfactory performance on three benchmarks.
Fashion products typically feature in compositions of a variety of styles at different clothing parts. In order to distinguish images of different fashion products, we need to extract both appearance (i.e., "how to describe") and localization (i.e.,"where to look") information, and their interactions. To this end, we propose a biologically inspired framework for image-based fashion product retrieval, which mimics the hypothesized twostream visual processing system of human brain. The proposed attentional heterogeneous bilinear network (AHBN) consists of two branches: a deep CNN branch to extract fine-grained appearance attributes and a fully convolutional branch to extract landmark localization information. A joint channel-wise attention mechanism is further applied to the extracted heterogeneous features to focus on important channels, followed by a compact bilinear pooling layer to model the interaction of the two streams. Our proposed framework achieves satisfactory performance on three image-based fashion product retrieval benchmarks.