GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks
This addresses the need for more reliable geospatial code generation in geosciences, though it is incremental as it builds on existing LLM fine-tuning techniques.
The paper tackles the problem of large language models struggling with geospatial code generation due to lack of domain knowledge, by introducing GeoCode-GPT-7B, a model fine-tuned from Code Llama-7B using new datasets and evaluation methods. Experimental results show it outperforms other models by 9.1% to 32.1% in multiple-choice accuracy, 1.7% to 25.4% in code summarization, and 1.2% to 25.1% in code generation.
The increasing demand for spatiotemporal data and modeling tasks in geosciences has made geospatial code generation technology a critical factor in enhancing productivity. Although large language models (LLMs) have demonstrated potential in code generation tasks, they often encounter issues such as refusal to code or hallucination in geospatial code generation due to a lack of domain-specific knowledge and code corpora. To address these challenges, this paper presents and open-sources the GeoCode-PT and GeoCode-SFT corpora, along with the GeoCode-Eval evaluation dataset. Additionally, by leveraging QLoRA and LoRA for pretraining and fine-tuning, we introduce GeoCode-GPT-7B, the first LLM focused on geospatial code generation, fine-tuned from Code Llama-7B. Furthermore, we establish a comprehensive geospatial code evaluation framework, incorporating option matching, expert validation, and prompt engineering scoring for LLMs, and systematically evaluate GeoCode-GPT-7B using the GeoCode-Eval dataset. Experimental results show that GeoCode-GPT outperforms other models in multiple-choice accuracy by 9.1% to 32.1%, in code summarization ability by 1.7% to 25.4%, and in code generation capability by 1.2% to 25.1%. This paper provides a solution and empirical validation for enhancing LLMs' performance in geospatial code generation, extends the boundaries of domain-specific model applications, and offers valuable insights into unlocking their potential in geospatial code generation.