Cleaning English Abstracts of Scientific Publications
This addresses a data quality issue for researchers and analysts using scientific abstracts in downstream tasks, but it is incremental as it builds on existing cleaning methods with a new model.
The paper tackled the problem of extraneous information in scientific abstracts, which distorts analyses like document similarity, by introducing an open-source language model that cleans them, resulting in improved similarity rankings and embedding information content.
Scientific abstracts are often used as proxies for the content and thematic focus of research publications. However, a significant share of published abstracts contains extraneous information-such as publisher copyright statements, section headings, author notes, registrations, and bibliometric or bibliographic metadata-that can distort downstream analyses, particularly those involving document similarity or textual embeddings. We introduce an open-source, easy-to-integrate language model designed to clean English-language scientific abstracts by automatically identifying and removing such clutter. We demonstrate that our model is both conservative and precise, alters similarity rankings of cleaned abstracts and improves information content of standard-length embeddings.