The Empowerment of Science of Science by Large Language Models: New Tools and Methods
It proposes incremental applications of existing LLM technologies to improve scientometric analysis for researchers and policymakers.
This paper reviews how large language models (LLMs) can enhance Science of Science (SciSci) by enabling new tools like AI agent-based scientific evaluation, research fronts detection, and knowledge graph building methods.
Large language models (LLMs) have exhibited exceptional capabilities in natural language understanding and generation, image recognition, and multimodal tasks, charting a course towards AGI and emerging as a central issue in the global technological race. This manuscript conducts a comprehensive review of the core technologies that support LLMs from a user standpoint, including prompt engineering, knowledge-enhanced retrieval augmented generation, fine tuning, pretraining, and tool learning. Additionally, it traces the historical development of Science of Science (SciSci) and presents a forward looking perspective on the potential applications of LLMs within the scientometric domain. Furthermore, it discusses the prospect of an AI agent based model for scientific evaluation, and presents new research fronts detection and knowledge graph building methods with LLMs.