GaLeNet: Multimodal Learning for Disaster Prediction, Management and Relief
This addresses the need for timely resource allocation in disaster management by providing a multimodal approach, though it appears incremental as it builds on existing multimodal methods for a specific domain.
The paper tackles the problem of assessing disaster damage severity by proposing GaLeNet, a multimodal framework that integrates pre-disaster images, weather data, and hurricane trajectory, demonstrating its effectiveness in experiments on two hurricanes and showing it can operate without post-disaster images to avoid delays.
After a natural disaster, such as a hurricane, millions are left in need of emergency assistance. To allocate resources optimally, human planners need to accurately analyze data that can flow in large volumes from several sources. This motivates the development of multimodal machine learning frameworks that can integrate multiple data sources and leverage them efficiently. To date, the research community has mainly focused on unimodal reasoning to provide granular assessments of the damage. Moreover, previous studies mostly rely on post-disaster images, which may take several days to become available. In this work, we propose a multimodal framework (GaLeNet) for assessing the severity of damage by complementing pre-disaster images with weather data and the trajectory of the hurricane. Through extensive experiments on data from two hurricanes, we demonstrate (i) the merits of multimodal approaches compared to unimodal methods, and (ii) the effectiveness of GaLeNet at fusing various modalities. Furthermore, we show that GaLeNet can leverage pre-disaster images in the absence of post-disaster images, preventing substantial delays in decision making.