SsciBERT: A Pre-trained Language Model for Social Science Texts
This addresses the need for researchers to quickly find relevant social science literature, though it is incremental as it adapts existing methods to a new domain.
The authors tackled the lack of a pre-trained language model for social science texts by proposing SsciBERT, trained on abstracts from SSCI journals, which achieved excellent performance on tasks like discipline classification and named entity recognition.
The academic literature of social sciences records human civilization and studies human social problems. With its large-scale growth, the ways to quickly find existing research on relevant issues have become an urgent demand for researchers. Previous studies, such as SciBERT, have shown that pre-training using domain-specific texts can improve the performance of natural language processing tasks. However, the pre-trained language model for social sciences is not available so far. In light of this, the present research proposes a pre-trained model based on the abstracts published in the Social Science Citation Index (SSCI) journals. The models, which are available on GitHub (https://github.com/S-T-Full-Text-Knowledge-Mining/SSCI-BERT), show excellent performance on discipline classification, abstract structure-function recognition, and named entity recognition tasks with the social sciences literature.