Quantifying the synthetic and real domain gap in aerial scene understanding
This work addresses the domain gap issue for researchers and practitioners in aerial scene understanding, providing insights to improve simulation fidelity and model generalization, though it is incremental in nature.
The paper tackled the problem of quantifying the gap between synthetic and real-world imagery in aerial scene understanding, finding that real-world scenes generally show higher consensus among vision transformers while synthetic scenes exhibit greater variability and challenge model adaptability.
Quantifying the gap between synthetic and real-world imagery is essential for improving both transformer-based models - that rely on large volumes of data - and datasets, especially in underexplored domains like aerial scene understanding where the potential impact is significant. This paper introduces a novel methodology for scene complexity assessment using Multi-Model Consensus Metric (MMCM) and depth-based structural metrics, enabling a robust evaluation of perceptual and structural disparities between domains. Our experimental analysis, utilizing real-world (Dronescapes) and synthetic (Skyscenes) datasets, demonstrates that real-world scenes generally exhibit higher consensus among state-of-the-art vision transformers, while synthetic scenes show greater variability and challenge model adaptability. The results underline the inherent complexities and domain gaps, emphasizing the need for enhanced simulation fidelity and model generalization. This work provides critical insights into the interplay between domain characteristics and model performance, offering a pathway for improved domain adaptation strategies in aerial scene understanding.