Cooperative Embeddings for Instance, Attribute and Category Retrieval
This addresses image retrieval for users needing multi-faceted similarity analysis, but it is incremental as it builds on existing embedding methods.
The paper tackles the problem of retrieving images based on instance, attribute, and category similarities by introducing a cooperative embedding that integrates these entities, and experiments show benefits for modeling multiple image similarities and discovering style evolution.
The goal of this paper is to retrieve an image based on instance, attribute and category similarity notions. Different from existing works, which usually address only one of these entities in isolation, we introduce a cooperative embedding to integrate them while preserving their specific level of semantic representation. An algebraic structure defines a superspace filled with instances. Attributes are axis-aligned to form subspaces, while categories influence the arrangement of similar instances. These relationships enable them to cooperate for their mutual benefits for image retrieval. We derive a proxy-based softmax embedding loss to learn simultaneously all similarity measures in both superspace and subspaces. We evaluate our model on datasets from two different domains. Experiments on image retrieval tasks show the benefits of the cooperative embeddings for modeling multiple image similarities, and for discovering style evolution of instances between- and within-categories.