Artificial Intuition: Efficient Classification of Scientific Abstracts
This addresses the problem of automating classification for research portfolio management, though it appears incremental as it builds on existing LLM capabilities.
The paper tackles the challenge of automatically classifying short scientific texts like grant abstracts by developing a novel approach that uses a Large Language Model to generate and assign coarse domain-specific labels, achieving results that are evaluated with new assessment tools alongside established metrics.
It is desirable to coarsely classify short scientific texts, such as grant or publication abstracts, for strategic insight or research portfolio management. These texts efficiently transmit dense information to experts possessing a rich body of knowledge to aid interpretation. Yet this task is remarkably difficult to automate because of brevity and the absence of context. To address this gap, we have developed a novel approach to generate and appropriately assign coarse domain-specific labels. We show that a Large Language Model (LLM) can provide metadata essential to the task, in a process akin to the augmentation of supplemental knowledge representing human intuition, and propose a workflow. As a pilot study, we use a corpus of award abstracts from the National Aeronautics and Space Administration (NASA). We develop new assessment tools in concert with established performance metrics.