WildFireCan-MMD: A Multimodal Dataset for Classification of User-Generated Content During Wildfires in Canada
This work addresses the need for localized, real-time information during Canadian wildfires, though it is incremental as it builds on existing multimodal datasets by focusing on a specific region.
The authors tackled the problem of extracting relevant insights from social media during wildfires by creating WildFireCan-MMD, a multimodal dataset of Canadian wildfire posts, and found that custom-trained models achieved an 84.48% f-score, outperforming zero-shot and baseline methods.
Rapid information access is vital during wildfires, yet traditional data sources are slow and costly. Social media offers real-time updates, but extracting relevant insights remains a challenge. In this work, we focus on multimodal wildfire social media data, which, although existing in current datasets, is currently underrepresented in Canadian contexts. We present WildFireCan-MMD, a new multimodal dataset of X posts from recent Canadian wildfires, annotated across twelve key themes. We evaluate zero-shot vision-language models on this dataset and compare their results with those of custom-trained and baseline classifiers. We show that while baseline methods and zero-shot prompting offer quick deployment, custom-trained models outperform them when labelled data is available. Our best-performing custom model reaches 84.48% f-score, outperforming VLMs and baseline classifiers. We also demonstrate how this model can be used to uncover trends during wildfires, through the collection and analysis of a large unlabeled dataset. Our dataset facilitates future research in wildfire response, and our findings highlight the importance of tailored datasets and task-specific training. Importantly, such datasets should be localized, as disaster response requirements vary across regions and contexts.