Encoder Fine-tuning with Stochastic Sampling Outperforms Open-weight GPT in Astronomy Knowledge Extraction
This work addresses the need for efficient information extraction in the rapidly growing field of astronomy, though it appears incremental as it builds on existing transformer models with a specific sampling technique.
The paper tackled the problem of automating knowledge extraction from astronomy literature by developing an encoder-based system that classifies telescope references, detects semantic attributes, and recognizes instrument mentions, and it significantly outperformed an open-weight GPT baseline.
Scientific literature in astronomy is rapidly expanding, making it increasingly important to automate the extraction of key entities and contextual information from research papers. In this paper, we present an encoder-based system for extracting knowledge from astronomy articles. Our objective is to develop models capable of classifying telescope references, detecting auxiliary semantic attributes, and recognizing instrument mentions from textual content. To this end, we implement a multi-task transformer-based system built upon the SciBERT model and fine-tuned for astronomy corpora classification. To carry out the fine-tuning, we stochastically sample segments from the training data and use majority voting over the test segments at inference time. Our system, despite its simplicity and low-cost implementation, significantly outperforms the open-weight GPT baseline.