Eliciting Disease Data from Wikipedia Articles
This addresses the need for policy makers and researchers to access timely, community-driven disease data, though it is incremental as it builds on existing internet-based surveillance methods.
The authors tackled the lack of real-time data repositories for disease outbreaks by developing a named-entity recognizer to extract case, death, and hospitalization counts from Wikipedia articles, achieving an F1 score of 0.753 and showing close alignment with ground truth data in a case study.
Traditional disease surveillance systems suffer from several disadvantages, including reporting lags and antiquated technology, that have caused a movement towards internet-based disease surveillance systems. Internet systems are particularly attractive for disease outbreaks because they can provide data in near real-time and can be verified by individuals around the globe. However, most existing systems have focused on disease monitoring and do not provide a data repository for policy makers or researchers. In order to fill this gap, we analyzed Wikipedia article content. We demonstrate how a named-entity recognizer can be trained to tag case counts, death counts, and hospitalization counts in the article narrative that achieves an F1 score of 0.753. We also show, using the 2014 West African Ebola virus disease epidemic article as a case study, that there are detailed time series data that are consistently updated that closely align with ground truth data. We argue that Wikipedia can be used to create the first community-driven open-source emerging disease detection, monitoring, and repository system.