Wiki-Quantities and Wiki-Measurements: Datasets of Quantities and their Measurement Context from Wikipedia
This addresses a data scarcity problem for researchers in natural language processing, especially in the natural and engineering sciences, by providing annotated datasets for quantity extraction, though it is incremental as it builds on existing resources like Wikipedia and Wikidata.
The authors tackled the lack of datasets for identifying quantities and their context in text by presenting Wiki-Quantities and Wiki-Measurements, two large datasets based on Wikipedia and Wikidata, with Wiki-Quantities containing over 1.2 million annotated quantities and Wiki-Measurements having 38,738 annotated quantities with measured entities and properties, achieving 100% and 84-94% correctness in manual validation, respectively.
To cope with the large number of publications, more and more researchers are automatically extracting data of interest using natural language processing methods based on supervised learning. Much data, especially in the natural and engineering sciences, is quantitative, but there is a lack of datasets for identifying quantities and their context in text. To address this issue, we present two large datasets based on Wikipedia and Wikidata: Wiki-Quantities is a dataset consisting of over 1.2 million annotated quantities in the English-language Wikipedia. Wiki-Measurements is a dataset of 38,738 annotated quantities in the English-language Wikipedia along with their respective measured entity, property, and optional qualifiers. Manual validation of 100 samples each of Wiki-Quantities and Wiki-Measurements found 100% and 84-94% correct, respectively. The datasets can be used in pipeline approaches to measurement extraction, where quantities are first identified and then their measurement context. To allow reproduction of this work using newer or different versions of Wikipedia and Wikidata, we publish the code used to create the datasets along with the data.