Enhance Image Classification via Inter-Class Image Mixup with Diffusion Model
This work addresses the challenge of effectively integrating text-to-image generative models into image classification tasks for domain-specific datasets, representing an incremental advancement in data augmentation techniques.
The paper tackled the problem of improving image classification by addressing the limitations of existing data augmentation methods, which struggle to generate images that are both faithful to foreground objects and diverse in background contexts for domain-specific concepts. The result was the introduction of Diff-Mix, an inter-class data augmentation method that achieved better balance and led to marked performance improvements across few-shot, conventional, and long-tail classification scenarios.
Text-to-image (T2I) generative models have recently emerged as a powerful tool, enabling the creation of photo-realistic images and giving rise to a multitude of applications. However, the effective integration of T2I models into fundamental image classification tasks remains an open question. A prevalent strategy to bolster image classification performance is through augmenting the training set with synthetic images generated by T2I models. In this study, we scrutinize the shortcomings of both current generative and conventional data augmentation techniques. Our analysis reveals that these methods struggle to produce images that are both faithful (in terms of foreground objects) and diverse (in terms of background contexts) for domain-specific concepts. To tackle this challenge, we introduce an innovative inter-class data augmentation method known as Diff-Mix (https://github.com/Zhicaiwww/Diff-Mix), which enriches the dataset by performing image translations between classes. Our empirical results demonstrate that Diff-Mix achieves a better balance between faithfulness and diversity, leading to a marked improvement in performance across diverse image classification scenarios, including few-shot, conventional, and long-tail classifications for domain-specific datasets.