MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction
This work addresses the need for better text mining tools in materials science, enabling more efficient extraction of knowledge from scientific literature to aid in materials discovery or optimization, though it is incremental as it adapts existing BERT methods to a specific domain.
The authors tackled the problem of suboptimal information extraction from materials science texts by developing MatSciBERT, a domain-specific language model trained on a large corpus of materials literature, which outperformed SciBERT on tasks like abstract classification, named entity recognition, and relation extraction.
An overwhelmingly large amount of knowledge in the materials domain is generated and stored as text published in peer-reviewed scientific literature. Recent developments in natural language processing, such as bidirectional encoder representations from transformers (BERT) models, provide promising tools to extract information from these texts. However, direct application of these models in the materials domain may yield suboptimal results as the models themselves may not be trained on notations and jargon that are specific to the domain. Here, we present a materials-aware language model, namely, MatSciBERT, which is trained on a large corpus of scientific literature published in the materials domain. We further evaluate the performance of MatSciBERT on three downstream tasks, namely, abstract classification, named entity recognition, and relation extraction, on different materials datasets. We show that MatSciBERT outperforms SciBERT, a language model trained on science corpus, on all the tasks. Further, we discuss some of the applications of MatSciBERT in the materials domain for extracting information, which can, in turn, contribute to materials discovery or optimization. Finally, to make the work accessible to the larger materials community, we make the pretrained and finetuned weights and the models of MatSciBERT freely accessible.