LTGC: Long-tail Recognition via Leveraging LLMs-driven Generated Content
It addresses data imbalance in long-tail recognition for computer vision applications, representing an incremental improvement with a novel generative approach.
The paper tackles long-tail recognition by using large language models to generate diverse tail-class content and fine-tuning with novel designs, achieving state-of-the-art performance on popular benchmarks.
Long-tail recognition is challenging because it requires the model to learn good representations from tail categories and address imbalances across all categories. In this paper, we propose a novel generative and fine-tuning framework, LTGC, to handle long-tail recognition via leveraging generated content. Firstly, inspired by the rich implicit knowledge in large-scale models (e.g., large language models, LLMs), LTGC leverages the power of these models to parse and reason over the original tail data to produce diverse tail-class content. We then propose several novel designs for LTGC to ensure the quality of the generated data and to efficiently fine-tune the model using both the generated and original data. The visualization demonstrates the effectiveness of the generation module in LTGC, which produces accurate and diverse tail data. Additionally, the experimental results demonstrate that our LTGC outperforms existing state-of-the-art methods on popular long-tailed benchmarks.