Multi-Modal Machine Learning for Flood Detection in News, Social Media and Satellite Sequences
This work addresses flood monitoring for disaster response by integrating multi-modal data, but it is incremental as it builds on existing fusion techniques and benchmarks.
The paper tackled flood detection by combining visual and textual information from news, social media, and satellite sequences, achieving F1-scores up to 82.63 for flood event prediction and around 58 for flood level estimation and satellite-based detection.
In this paper we present our methods for the MediaEval 2019 Mul-timedia Satellite Task, which is aiming to extract complementaryinformation associated with adverse events from Social Media andsatellites. For the first challenge, we propose a framework jointly uti-lizing colour, object and scene-level information to predict whetherthe topic of an article containing an image is a flood event or not.Visual features are combined using early and late fusion techniquesachieving an average F1-score of82.63,82.40,81.40and76.77. Forthe multi-modal flood level estimation, we rely on both visualand textual information achieving an average F1-score of58.48and46.03, respectively. Finally, for the flooding detection in time-based satellite image sequences we used a combination of classicalcomputer-vision and machine learning approaches achieving anaverage F1-score of58.82%