Deep Multiple Instance Learning for Zero-shot Image Tagging
This addresses the scalability issue in zero-shot recognition for multi-label scenarios, enabling tagging of multiple unseen objects in images, which is incremental as it builds on existing deep learning and MIL methods.
The paper tackles the problem of multi-label zero-shot image tagging, where multiple unseen objects need to be identified in an image, by proposing the first end-to-end trainable deep Multiple Instance Learning framework, achieving superior performance on the NUS-WIDE dataset across conventional, zero-shot, and generalized zero-shot tagging tasks.
In-line with the success of deep learning on traditional recognition problem, several end-to-end deep models for zero-shot recognition have been proposed in the literature. These models are successful to predict a single unseen label given an input image, but does not scale to cases where multiple unseen objects are present. In this paper, we model this problem within the framework of Multiple Instance Learning (MIL). To the best of our knowledge, we propose the first end-to-end trainable deep MIL framework for the multi-label zero-shot tagging problem. Due to its novel design, the proposed framework has several interesting features: (1) Unlike previous deep MIL models, it does not use any off-line procedure (e.g., Selective Search or EdgeBoxes) for bag generation. (2) During test time, it can process any number of unseen labels given their semantic embedding vectors. (3) Using only seen labels per image as weak annotation, it can produce a bounding box for each predicted labels. We experiment with the NUS-WIDE dataset and achieve superior performance across conventional, zero-shot and generalized zero-shot tagging tasks.