Not just about size - A Study on the Role of Distributed Word Representations in the Analysis of Scientific Publications
This work addresses the challenge of optimizing word embeddings for analyzing scientific texts, which is incremental as it builds on existing methods by comparing domain-specific and general corpora.
The study investigated the trade-off between corpus specificity and volume for generating word embeddings from scientific publications, finding that domain-specific corpora effectively capture domain semantics while general corpora require large volumes to achieve comparable results.
The emergence of knowledge graphs in the scholarly communication domain and recent advances in artificial intelligence and natural language processing bring us closer to a scenario where intelligent systems can assist scientists over a range of knowledge-intensive tasks. In this paper we present experimental results about the generation of word embeddings from scholarly publications for the intelligent processing of scientific texts extracted from SciGraph. We compare the performance of domain-specific embeddings with existing pre-trained vectors generated from very large and general purpose corpora. Our results suggest that there is a trade-off between corpus specificity and volume. Embeddings from domain-specific scientific corpora effectively capture the semantics of the domain. On the other hand, obtaining comparable results through general corpora can also be achieved, but only in the presence of very large corpora of well formed text. Furthermore, We also show that the degree of overlapping between knowledge areas is directly related to the performance of embeddings in domain evaluation tasks.