HiGFA: Hierarchical Guidance for Fine-grained Data Augmentation with Diffusion Models
This addresses the problem of generating high-fidelity synthetic data for fine-grained visual classification tasks, offering a domain-specific incremental improvement over standard methods.
The paper tackles the challenge of using diffusion models for fine-grained data augmentation by proposing HiGFA, which hierarchically guides the sampling process with text, contour, and classifier signals; experiments on FGVC datasets show it generates diverse and faithful synthetic images that improve classifier performance.
Generative diffusion models show promise for data augmentation. However, applying them to fine-grained tasks presents a significant challenge: ensuring synthetic images accurately capture the subtle, category-defining features critical for high fidelity. Standard approaches, such as text-based Classifier-Free Guidance (CFG), often lack the required specificity, potentially generating misleading examples that degrade fine-grained classifier performance. To address this, we propose Hierarchically Guided Fine-grained Augmentation (HiGFA). HiGFA leverages the temporal dynamics of the diffusion sampling process. It employs strong text and transformed contour guidance with fixed strengths in the early-to-mid sampling stages to establish overall scene, style, and structure. In the final sampling stages, HiGFA activates a specialized fine-grained classifier guidance and dynamically modulates the strength of all guidance signals based on prediction confidence. This hierarchical, confidence-driven orchestration enables HiGFA to generate diverse yet faithful synthetic images by intelligently balancing global structure formation with precise detail refinement. Experiments on several FGVC datasets demonstrate the effectiveness of HiGFA.