Learning Subclass Representations for Visually-varied Image Classification
This work addresses the challenge of image classification for visually varied categories, such as social media photos, by leveraging tag co-occurrence to create more stable subclasses, offering a domain-specific improvement.
The paper tackles the problem of classifying visually diverse images by introducing a subclass-representation approach that uses co-occurring tags to define subclasses, then projects images into this subclass space for classification. The method was evaluated on 2 million Flickr photos across 10 classes and showed improved performance for visually diverse classes compared to direct top-level modeling.
In this paper, we present a subclass-representation approach that predicts the probability of a social image belonging to one particular class. We explore the co-occurrence of user-contributed tags to find subclasses with a strong connection to the top level class. We then project each image on to the resulting subclass space to generate a subclass representation for the image. The novelty of the approach is that subclass representations make use of not only the content of the photos themselves, but also information on the co-occurrence of their tags, which determines membership in both subclasses and top-level classes. The novelty is also that the images are classified into smaller classes, which have a chance of being more visually stable and easier to model. These subclasses are used as a latent space and images are represented in this space by their probability of relatedness to all of the subclasses. In contrast to approaches directly modeling each top-level class based on the image content, the proposed method can exploit more information for visually diverse classes. The approach is evaluated on a set of $2$ million photos with 10 classes, released by the Multimedia 2013 Yahoo! Large-scale Flickr-tag Image Classification Grand Challenge. Experiments show that the proposed system delivers sound performance for visually diverse classes compared with methods that directly model top classes.