Effective Damage Data Generation by Fusing Imagery with Human Knowledge Using Vision-Language Models
This work addresses data scarcity and generalization issues for disaster response teams, though it appears incremental in applying existing VLM techniques to a specific domain.
The paper tackled the problem of assessing damages in humanitarian assistance and disaster response by addressing data imbalance and labeling inaccuracies in deep learning approaches, resulting in improved classification of structural damage scenes through effective data generation using vision-language models.
It is of crucial importance to assess damages promptly and accurately in humanitarian assistance and disaster response (HADR). Current deep learning approaches struggle to generalize effectively due to the imbalance of data classes, scarcity of moderate damage examples, and human inaccuracy in pixel labeling during HADR situations. To accommodate for these limitations and exploit state-of-the-art techniques in vision-language models (VLMs) to fuse imagery with human knowledge understanding, there is an opportunity to generate a diversified set of image-based damage data effectively. Our initial experimental results suggest encouraging data generation quality, which demonstrates an improvement in classifying scenes with different levels of structural damage to buildings, roads, and infrastructures.