Cross-Domain Image Retrieval with Attention Modeling
This addresses a practical problem for e-commerce users and platforms by enabling more effective image-based product searches, though it is incremental as it builds on existing deep learning techniques.
The paper tackles the problem of cross-domain image retrieval, where smartphone photos are used to search for products on e-commerce sites, by proposing novel deep neural network architectures (TagYNet and CtxYNet) to locate attention in images, resulting in significant improvements in retrieval accuracy and efficiency over existing methods.
With the proliferation of e-commerce websites and the ubiquitousness of smart phones, cross-domain image retrieval using images taken by smart phones as queries to search products on e-commerce websites is emerging as a popular application. One challenge of this task is to locate the attention of both the query and database images. In particular, database images, e.g. of fashion products, on e-commerce websites are typically displayed with other accessories, and the images taken by users contain noisy background and large variations in orientation and lighting. Consequently, their attention is difficult to locate. In this paper, we exploit the rich tag information available on the e-commerce websites to locate the attention of database images. For query images, we use each candidate image in the database as the context to locate the query attention. Novel deep convolutional neural network architectures, namely TagYNet and CtxYNet, are proposed to learn the attention weights and then extract effective representations of the images. Experimental results on public datasets confirm that our approaches have significant improvement over the existing methods in terms of the retrieval accuracy and efficiency.