Parametric Classification for Generalized Category Discovery: A Baseline Study
This work addresses the challenge of discovering novel categories in unlabelled datasets for machine learning researchers, offering an incremental improvement by refining parametric classifiers in GCD.
The study tackled the problem of parametric classifiers overfitting to seen categories in Generalized Category Discovery (GCD) by identifying unreliable pseudo-labels and prediction biases, and proposed a simple parametric method with entropy regularization that achieves state-of-the-art performance on multiple benchmarks with strong robustness to unknown class numbers.
Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. Previous studies argued that parametric classifiers are prone to overfitting to seen categories, and endorsed using a non-parametric classifier formed with semi-supervised k-means. However, in this study, we investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We demonstrate that two prediction biases exist: the classifier tends to predict seen classes more often, and produces an imbalanced distribution across seen and novel categories. Based on these findings, we propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers. We hope the investigation and proposed simple framework can serve as a strong baseline to facilitate future studies in this field. Our code is available at: https://github.com/CVMI-Lab/SimGCD.