ZiGong 1.0: A Large Language Model for Financial Credit
This work addresses the need for more reliable AI in financial credit assessment, offering a domain-specific solution that is incremental in nature.
The paper tackles the problem of suboptimal performance of large language models in financial credit assessment by proposing ZiGong, a Mistral-based model enhanced with multi-task supervised fine-tuning and a novel data pruning method to reduce hallucinations, resulting in significant improvements in robustness and prediction accuracy in real-world financial scenarios.
Large Language Models (LLMs) have demonstrated strong performance across various general Natural Language Processing (NLP) tasks. However, their effectiveness in financial credit assessment applications remains suboptimal, primarily due to the specialized financial expertise required for these tasks. To address this limitation, we propose ZiGong, a Mistral-based model enhanced through multi-task supervised fine-tuning. To specifically combat model hallucination in financial contexts, we introduce a novel data pruning methodology. Our approach utilizes a proxy model to score training samples, subsequently combining filtered data with original datasets for model training. This data refinement strategy effectively reduces hallucinations in LLMs while maintaining reliability in downstream financial applications. Experimental results show our method significantly enhances model robustness and prediction accuracy in real-world financial scenarios.