SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery
This work addresses the challenge of scalable and generalizable category discovery for computer vision applications, representing an incremental improvement over existing methods.
The paper tackles the problem of Generalized Category Discovery (GCD) by introducing SEAL, a semantic-aware hierarchical learning framework that categorizes unlabeled images into known or unknown classes, achieving state-of-the-art performance on fine-grained benchmarks like SSB, Oxford-Pet, and Herbarium19.
This paper investigates the problem of Generalized Category Discovery (GCD). Given a partially labelled dataset, GCD aims to categorize all unlabelled images, regardless of whether they belong to known or unknown classes. Existing approaches typically depend on either single-level semantics or manually designed abstract hierarchies, which limit their generalizability and scalability. To address these limitations, we introduce a SEmantic-aware hierArchical Learning framework (SEAL), guided by naturally occurring and easily accessible hierarchical structures. Within SEAL, we propose a Hierarchical Semantic-Guided Soft Contrastive Learning approach that exploits hierarchical similarity to generate informative soft negatives, addressing the limitations of conventional contrastive losses that treat all negatives equally. Furthermore, a Cross-Granularity Consistency (CGC) module is designed to align the predictions from different levels of granularity. SEAL consistently achieves state-of-the-art performance on fine-grained benchmarks, including the SSB benchmark, Oxford-Pet, and the Herbarium19 dataset, and further demonstrates generalization on coarse-grained datasets. Project page: https://visual-ai.github.io/seal/