Agentic AI for Improving Precision in Identifying Contributions to Sustainable Development Goals
This provides a scalable framework for improving SDG-related research metrics and institutional reporting, addressing a specific need for research institutions, but it is incremental as it applies existing LLM methods to a new domain.
This study tackled the problem of accurately assessing research contributions to Sustainable Development Goals (SDGs) by addressing the low precision of keyword-based methods, and demonstrated that small, locally-hosted autoregressive LLMs can differentiate genuine contributions from incidental keyword matches in academic abstracts.
As research institutions increasingly commit to supporting the United Nations' Sustainable Development Goals (SDGs), there is a pressing need to accurately assess their research output against these goals. Current approaches, primarily reliant on keyword-based Boolean search queries, conflate incidental keyword matches with genuine contributions, reducing retrieval precision and complicating benchmarking efforts. This study investigates the application of autoregressive Large Language Models (LLMs) as evaluation agents to identify relevant scholarly contributions to SDG targets in scholarly publications. Using a dataset of academic abstracts retrieved via SDG-specific keyword queries, we demonstrate that small, locally-hosted LLMs can differentiate semantically relevant contributions to SDG targets from documents retrieved due to incidental keyword matches, addressing the limitations of traditional methods. By leveraging the contextual understanding of LLMs, this approach provides a scalable framework for improving SDG-related research metrics and informing institutional reporting.