FloodBrain: Flood Disaster Reporting by Web-based Retrieval Augmented Generation with an LLM
This work addresses the need for fast and reliable disaster reporting for humanitarian coordination, though it is incremental as it applies existing RAG methods to a specific domain.
The authors tackled the problem of generating accurate flood disaster impact reports by developing FloodBrain, a web-based retrieval-augmented generation pipeline that reduces LLM hallucinations, and found it produced reports with scores correlating well with human evaluations.
Fast disaster impact reporting is crucial in planning humanitarian assistance. Large Language Models (LLMs) are well known for their ability to write coherent text and fulfill a variety of tasks relevant to impact reporting, such as question answering or text summarization. However, LLMs are constrained by the knowledge within their training data and are prone to generating inaccurate, or "hallucinated", information. To address this, we introduce a sophisticated pipeline embodied in our tool FloodBrain (floodbrain.com), specialized in generating flood disaster impact reports by extracting and curating information from the web. Our pipeline assimilates information from web search results to produce detailed and accurate reports on flood events. We test different LLMs as backbones in our tool and compare their generated reports to human-written reports on different metrics. Similar to other studies, we find a notable correlation between the scores assigned by GPT-4 and the scores given by human evaluators when comparing our generated reports to human-authored ones. Additionally, we conduct an ablation study to test our single pipeline components and their relevancy for the final reports. With our tool, we aim to advance the use of LLMs for disaster impact reporting and reduce the time for coordination of humanitarian efforts in the wake of flood disasters.