Diffusion Features to Bridge Domain Gap for Semantic Segmentation
This work addresses domain gap issues in semantic segmentation for computer vision applications, representing an incremental improvement by adapting existing diffusion models.
The paper tackles cross-domain semantic segmentation by leveraging pre-trained diffusion models to extract and fuse features, proposing DIFF as a backbone and a training framework that implicitly learns from text-to-image generation, achieving state-of-the-art results in domain generalization benchmarks.
Pre-trained diffusion models have demonstrated remarkable proficiency in synthesizing images across a wide range of scenarios with customizable prompts, indicating their effective capacity to capture universal features. Motivated by this, our study delves into the utilization of the implicit knowledge embedded within diffusion models to address challenges in cross-domain semantic segmentation. This paper investigates the approach that leverages the sampling and fusion techniques to harness the features of diffusion models efficiently. We propose DIffusion Feature Fusion (DIFF) as a backbone use for extracting and integrating effective semantic representations through the diffusion process. By leveraging the strength of text-to-image generation capability, we introduce a new training framework designed to implicitly learn posterior knowledge from it. Through rigorous evaluation in the contexts of domain generalization semantic segmentation, we establish that our methodology surpasses preceding approaches in mitigating discrepancies across distinct domains and attains the state-of-the-art (SOTA) benchmark.