Towards Efficient Large Language Models for Scientific Text: A Review
It addresses the high computational costs of LLMs for researchers and practitioners in scientific domains, but is incremental as it compiles existing approaches without new results.
The paper reviews methods to make large language models (LLMs) more efficient for scientific text, summarizing advances in model size reduction and data quality enhancement to develop affordable AI solutions for science.
Large language models (LLMs) have ushered in a new era for processing complex information in various fields, including science. The increasing amount of scientific literature allows these models to acquire and understand scientific knowledge effectively, thus improving their performance in a wide range of tasks. Due to the power of LLMs, they require extremely expensive computational resources, intense amounts of data, and training time. Therefore, in recent years, researchers have proposed various methodologies to make scientific LLMs more affordable. The most well-known approaches align in two directions. It can be either focusing on the size of the models or enhancing the quality of data. To date, a comprehensive review of these two families of methods has not yet been undertaken. In this paper, we (I) summarize the current advances in the emerging abilities of LLMs into more accessible AI solutions for science, and (II) investigate the challenges and opportunities of developing affordable solutions for scientific domains using LLMs.