Enhancing Features in Long-tailed Data Using Large Vision Model
This addresses long-tailed data issues in computer vision, but it is incremental as it adapts existing foundation model techniques to a specific domain.
The paper tackles the problem of long-tailed recognition by using large vision models to enhance features without language data, achieving competitive results on ImageNet-LT and iNaturalist2018 benchmarks.
Language-based foundation models, such as large language models (LLMs) or large vision-language models (LVLMs), have been widely studied in long-tailed recognition. However, the need for linguistic data is not applicable to all practical tasks. In this study, we aim to explore using large vision models (LVMs) or visual foundation models (VFMs) to enhance long-tailed data features without any language information. Specifically, we extract features from the LVM and fuse them with features in the baseline network's map and latent space to obtain the augmented features. Moreover, we design several prototype-based losses in the latent space to further exploit the potential of the augmented features. In the experimental section, we validate our approach on two benchmark datasets: ImageNet-LT and iNaturalist2018.