RATE: An LLM-Powered Retrieval Augmented Generation Technology-Extraction Pipeline
This work addresses the problem of creating technology maps for decision-making, specifically for researchers and analysts in fields like Brain-Computer Interfaces and Extended Reality, but it is incremental as it adapts existing LLM and RAG techniques to a new application.
The paper tackles automated technology extraction from scientific literature by introducing RATE, an LLM-based pipeline that combines Retrieval Augmented Generation with multi-definition validation, achieving an F1-score of 91.27% on a BCI-XR dataset, significantly outperforming a BERT-based method at 53.73%.
In an era of radical technology transformations, technology maps play a crucial role in enhancing decision making. These maps heavily rely on automated methods of technology extraction. This paper introduces Retrieval Augmented Technology Extraction (RATE), a Large Language Model (LLM) based pipeline for automated technology extraction from scientific literature. RATE combines Retrieval Augmented Generation (RAG) with multi-definition LLM-based validation. This hybrid method results in high recall in candidate generation alongside with high precision in candidate filtering. While the pipeline is designed to be general and widely applicable, we demonstrate its use on 678 research articles focused on Brain-Computer Interfaces (BCIs) and Extended Reality (XR) as a case study. Consequently, The validated technology terms by RATE were mapped into a co-occurrence network, revealing thematic clusters and structural features of the research landscape. For the purpose of evaluation, a gold standard dataset of technologies in 70 selected random articles had been curated by the experts. In addition, a technology extraction model based on Bidirectional Encoder Representations of Transformers (BERT) was used as a comparative method. RATE achieved F1-score of 91.27%, Significantly outperforming BERT with F1-score of 53.73%. Our findings highlight the promise of definition-driven LLM methods for technology extraction and mapping. They also offer new insights into emerging trends within the BCI-XR field. The source code is available https://github.com/AryaAftab/RATE