Late or Earlier Information Fusion from Depth and Spectral Data? Large-Scale Digital Surface Model Refinement by Hybrid-cGAN
This work addresses DSM refinement for remote sensing applications, but it is incremental as it builds on existing cGAN methods with a hybrid approach.
The paper tackled the problem of refining digital surface models (DSMs) by fusing depth and spectral data, using a Hybrid-cGAN to blend information earlier in the network, resulting in improved building boundaries and more rectilinear forms.
We present the workflow of a DSM refinement methodology using a Hybrid-cGAN where the generative part consists of two encoders and a common decoder which blends the spectral and height information within one network. The inputs to the Hybrid-cGAN are single-channel photogrammetric DSMs with continuous values and single-channel pan-chromatic (PAN) half-meter resolution satellite images. Experimental results demonstrate that the earlier information fusion from data with different physical meanings helps to propagate fine details and complete an inaccurate or missing 3D information about building forms. Moreover, it improves the building boundaries making them more rectilinear.