MultiEarth 2022 -- The Champion Solution for the Matrix Completion Challenge via Multimodal Regression and Generation
This work addresses data sparsity challenges in earth observation for satellite monitoring applications, representing an incremental improvement in a specific competition setting.
The paper tackled the problem of missing satellite observations due to complex sensing conditions by proposing an adaptive real-time multimodal regression and generation framework, achieving superior performance on unseen test queries with an LPIPS of 0.2226, a PSNR of 123.0372, and an SSIM of 0.6347 in the MultiEarth Matrix Completion Challenge.
Earth observation satellites have been continuously monitoring the earth environment for years at different locations and spectral bands with different modalities. Due to complex satellite sensing conditions (e.g., weather, cloud, atmosphere, orbit), some observations for certain modalities, bands, locations, and times may not be available. The MultiEarth Matrix Completion Challenge in CVPR 2022 [1] provides the multimodal satellite data for addressing such data sparsity challenges with the Amazon Rainforest as the region of interest. This work proposes an adaptive real-time multimodal regression and generation framework and achieves superior performance on unseen test queries in this challenge with an LPIPS of 0.2226, a PSNR of 123.0372, and an SSIM of 0.6347.