Dataset Mention Extraction in Scientific Articles Using Bi-LSTM-CRF Model
This addresses the issue of non-standard dataset citation in science, which hinders tracking and reproducibility, but is incremental as it applies an existing method to a specific domain.
The paper tackles the problem of automatically extracting dataset mentions from scientific articles to track their usage, achieving an F1 score of 0.885 on social science articles from the Rich Context Dataset.
Datasets are critical for scientific research, playing an important role in replication, reproducibility, and efficiency. Researchers have recently shown that datasets are becoming more important for science to function properly, even serving as artifacts of study themselves. However, citing datasets is not a common or standard practice in spite of recent efforts by data repositories and funding agencies. This greatly affects our ability to track their usage and importance. A potential solution to this problem is to automatically extract dataset mentions from scientific articles. In this work, we propose to achieve such extraction by using a neural network based on a Bi-LSTM-CRF architecture. Our method achieves F1 = 0.885 in social science articles released as part of the Rich Context Dataset. We discuss the limitations of the current datasets and propose modifications to the model to be done in the future.