Enhancing Ground-to-Aerial Image Matching for Visual Misinformation Detection Using Semantic Segmentation
This addresses the need for reliable geolocation in journalism, forensics, and Earth observation, but is incremental as it builds on existing SOTA approaches.
The study tackled the problem of matching ground-view images to satellite images without GPS data for detecting visual misinformation, achieving up to 9.8% improvement over prior methods on the CVUSA dataset.
The recent advancements in generative AI techniques, which have significantly increased the online dissemination of altered images and videos, have raised serious concerns about the credibility of digital media available on the Internet and distributed through information channels and social networks. This issue particularly affects domains that rely heavily on trustworthy data, such as journalism, forensic analysis, and Earth observation. To address these concerns, the ability to geolocate a non-geo-tagged ground-view image without external information, such as GPS coordinates, has become increasingly critical. This study tackles the challenge of linking a ground-view image, potentially exhibiting varying fields of view (FoV), to its corresponding satellite image without the aid of GPS data. To achieve this, we propose a novel four-stream Siamese-like architecture, the Quadruple Semantic Align Net (SAN-QUAD), which extends previous state-of-the-art (SOTA) approaches by leveraging semantic segmentation applied to both ground and satellite imagery. Experimental results on a subset of the CVUSA dataset demonstrate significant improvements of up to 9.8% over prior methods across various FoV settings.