Visually Consistent Hierarchical Image Classification
This work addresses hierarchical classification for computer vision applications, offering an incremental improvement by focusing on visual consistency without external semantic supervision.
The paper tackled the problem of hierarchical image classification by addressing visual inconsistency across taxonomy levels, proposing a method that enforces internal visual consistency through intra-image segmentation, which outperformed zero-shot CLIP and state-of-the-art baselines with higher accuracy and more consistent predictions.
Hierarchical classification predicts labels across multiple levels of a taxonomy, e.g., from coarse-level 'Bird' to mid-level 'Hummingbird' to fine-level 'Green hermit', allowing flexible recognition under varying visual conditions. It is commonly framed as multiple single-level tasks, but each level may rely on different visual cues: Distinguishing 'Bird' from 'Plant' relies on global features like feathers or leaves, while separating 'Anna's hummingbird' from 'Green hermit' requires local details such as head coloration. Prior methods improve accuracy using external semantic supervision, but such statistical learning criteria fail to ensure consistent visual grounding at test time, resulting in incorrect hierarchical classification. We propose, for the first time, to enforce internal visual consistency by aligning fine-to-coarse predictions through intra-image segmentation. Our method outperforms zero-shot CLIP and state-of-the-art baselines on hierarchical classification benchmarks, achieving both higher accuracy and more consistent predictions. It also improves internal image segmentation without requiring pixel-level annotations.