Image Classification Using a Diffusion Model as a Pre-Training Model
This work addresses the problem of limited labeled data for medical image classification, though it is incremental as it builds on existing diffusion and self-supervised learning methods.
The paper tackles the challenge of requiring large-scale labeled datasets for image classification by proposing a diffusion model with a representation-conditioning mechanism, achieving improvements of +6.15% in accuracy and +13.60% in F1-score over DINOv2 in zero-shot hematoma detection.
In this paper, we propose a diffusion model that integrates a representation-conditioning mechanism, where the representations derived from a Vision Transformer (ViT) are used to condition the internal process of a Transformer-based diffusion model. This approach enables representation-conditioned data generation, addressing the challenge of requiring large-scale labeled datasets by leveraging self-supervised learning on unlabeled data. We evaluate our method through a zero-shot classification task for hematoma detection in brain imaging. Compared to the strong contrastive learning baseline, DINOv2, our method achieves a notable improvement of +6.15% in accuracy and +13.60% in F1-score, demonstrating its effectiveness in image classification.