Visualizing and Describing Fine-grained Categories as Textures
This work addresses the challenge of explaining subtle differences in species for fine-grained visual categorization, though it is incremental as it builds on existing texture-based methods.
The paper tackles the problem of describing fine-grained visual categories by their textural content, visualizing 'maximal images' that maximize class probability in texture-based networks and automatically generating texture attribute descriptions, with models trained on an extended DTD dataset.
We analyze how categories from recent FGVC challenges can be described by their textural content. The motivation is that subtle differences between species of birds or butterflies can often be described in terms of the texture associated with them and that several top-performing networks are inspired by texture-based representations. These representations are characterized by orderless pooling of second-order filter activations such as in bilinear CNNs and the winner of the iNaturalist 2018 challenge. Concretely, for each category we (i) visualize the "maximal images" by obtaining inputs x that maximize the probability of the particular class according to a texture-based deep network, and (ii) automatically describe the maximal images using a set of texture attributes. The models for texture captioning were trained on our ongoing efforts on collecting a dataset of describable textures building on the DTD dataset. These visualizations indicate what aspects of the texture is most discriminative for each category while the descriptions provide a language-based explanation of the same.