Health Sentinel: An AI Pipeline For Real-time Disease Outbreak Detection
This work addresses the challenge of timely disease surveillance for public health authorities by automating the screening of online media, though it is incremental as it combines existing ML and non-ML methods.
The authors tackled the problem of early disease outbreak detection by proposing Health Sentinel, an AI pipeline that processes online news articles to extract health events, resulting in the identification of over 95,000 unique health events and over 3,500 potential outbreaks from over 300 million articles processed.
Early detection of disease outbreaks is crucial to ensure timely intervention by the health authorities. Due to the challenges associated with traditional indicator-based surveillance, monitoring informal sources such as online media has become increasingly popular. However, owing to the number of online articles getting published everyday, manual screening of the articles is impractical. To address this, we propose Health Sentinel. It is a multi-stage information extraction pipeline that uses a combination of ML and non-ML methods to extract events-structured information concerning disease outbreaks or other unusual health events-from online articles. The extracted events are made available to the Media Scanning and Verification Cell (MSVC) at the National Centre for Disease Control (NCDC), Delhi for analysis, interpretation and further dissemination to local agencies for timely intervention. From April 2022 till date, Health Sentinel has processed over 300 million news articles and identified over 95,000 unique health events across India of which over 3,500 events were shortlisted by the public health experts at NCDC as potential outbreaks.