A Generalized Multi-Task Learning Approach to Stereo DSM Filtering in Urban Areas
This work addresses the need for accurate urban height maps for applications like disaster management and city planning, representing an incremental improvement over existing CNN-based filtering approaches.
The paper tackled the problem of filtering noisy and vegetation-distorted stereo digital surface models (DSMs) in urban areas by proposing a modular multi-task learning framework with shared encoders and task-specific decoders, which consistently outperformed state-of-the-art methods on common data and generalized well to a new dataset.
City models and height maps of urban areas serve as a valuable data source for numerous applications, such as disaster management or city planning. While this information is not globally available, it can be substituted by digital surface models (DSMs), automatically produced from inexpensive satellite imagery. However, stereo DSMs often suffer from noise and blur. Furthermore, they are heavily distorted by vegetation, which is of lesser relevance for most applications. Such basic models can be filtered by convolutional neural networks (CNNs), trained on labels derived from digital elevation models (DEMs) and 3D city models, in order to obtain a refined DSM. We propose a modular multi-task learning concept that consolidates existing approaches into a generalized framework. Our encoder-decoder models with shared encoders and multiple task-specific decoders leverage roof type classification as a secondary task and multiple objectives including a conditional adversarial term. The contributing single-objective losses are automatically weighted in the final multi-task loss function based on learned uncertainty estimates. We evaluated the performance of specific instances of this family of network architectures. Our method consistently outperforms the state of the art on common data, both quantitatively and qualitatively, and generalizes well to a new dataset of an independent study area.