AdaptaGen: Domain-Specific Image Generation through Hierarchical Semantic Optimization Framework
This work addresses the problem of generating accurate and diverse images for specialized domains, which is incremental as it builds on existing prompt engineering and adaptation techniques.
The paper tackles the problem of domain-specific image generation by addressing semantic inaccuracies and hallucinations in existing methods, proposing AdaptaGen, a hierarchical semantic optimization framework that integrates prompt optimization and cross-modal adaptation. Experimental results show it achieves superior performance across 40 categories using only 16 images per category, with significant improvements in image quality, diversity, and semantic consistency.
Domain-specific image generation aims to produce high-quality visual content for specialized fields while ensuring semantic accuracy and detail fidelity. However, existing methods exhibit two critical limitations: First, current approaches address prompt engineering and model adaptation separately, overlooking the inherent dependence between semantic understanding and visual representation in specialized domains. Second, these techniques inadequately incorporate domain-specific semantic constraints during content synthesis, resulting in generation outcomes that exhibit hallucinations and semantic deviations. To tackle these issues, we propose AdaptaGen, a hierarchical semantic optimization framework that integrates matrix-based prompt optimization with multi-perspective understanding, capturing comprehensive semantic relationships from both global and local perspectives. To mitigate hallucinations in specialized domains, we design a cross-modal adaptation mechanism, which, when combined with intelligent content synthesis, enables preserving core thematic elements while incorporating diverse details across images. Additionally, we introduce a two-phase caption semantic transformation during the generation phase. This approach maintains semantic coherence while enhancing visual diversity, ensuring the generated images adhere to domain-specific constraints. Experimental results confirm our approach's effectiveness, with our framework achieving superior performance across 40 categories from diverse datasets using only 16 images per category, demonstrating significant improvements in image quality, diversity, and semantic consistency.