Large-Scale Attribute-Object Compositions
This addresses a computer vision problem for researchers and practitioners needing robust composition recognition, but it appears incremental as it builds on existing composition learning with a larger dataset and noisy supervision.
The paper tackles the problem of predicting attribute-object compositions from images, particularly generalizing to unseen compositions not in training data, using a large-scale dataset of hundreds of thousands of compositions with noisy Instagram hashtags as weak supervision. The results show that learning to compose classifiers outperforms late fusion of individual predictions, especially for unseen pairs.
We study the problem of learning how to predict attribute-object compositions from images, and its generalization to unseen compositions missing from the training data. To the best of our knowledge, this is a first large-scale study of this problem, involving hundreds of thousands of compositions. We train our framework with images from Instagram using hashtags as noisy weak supervision. We make careful design choices for data collection and modeling, in order to handle noisy annotations and unseen compositions. Finally, extensive evaluations show that learning to compose classifiers outperforms late fusion of individual attribute and object predictions, especially in the case of unseen attribute-object pairs.