Building a Massive Corpus for Named Entity Recognition using Free Open Data Sources
This addresses the need for large, cost-effective datasets for NER, which is incremental as it automates dataset creation using existing open data.
The paper tackles the problem of expensive human-annotated datasets for Named Entity Recognition (NER) by proposing a method to automatically generate labeled datasets from public sources like DBpedia and Wikipedia, resulting in a dataset with millions of labeled sentences that helps build better NER predictors.
With the recent progress in machine learning, boosted by techniques such as deep learning, many tasks can be successfully solved once a large enough dataset is available for training. Nonetheless, human-annotated datasets are often expensive to produce, especially when labels are fine-grained, as is the case of Named Entity Recognition (NER), a task that operates with labels on a word-level. In this paper, we propose a method to automatically generate labeled datasets for NER from public data sources by exploiting links and structured data from DBpedia and Wikipedia. Due to the massive size of these data sources, the resulting dataset -- SESAME Available at https://sesame-pt.github.io -- is composed of millions of labeled sentences. We detail the method to generate the dataset, report relevant statistics, and design a baseline using a neural network, showing that our dataset helps building better NER predictors.