LLMs Meet Finance: Fine-Tuning Foundation Models for the Open FinLLM Leaderboard
This work addresses financial applications for AI practitioners, but it is incremental as it builds on existing models and methods.
This paper tackled the problem of applying large language models to financial tasks by fine-tuning foundation models like Qwen2.5 and Deepseek-R1 using techniques such as SFT, DPO, and RL, resulting in substantial performance gains across various financial tasks and measurement of data scaling laws in finance.
This paper investigates the application of large language models (LLMs) to financial tasks. We fine-tuned foundation models using the Open FinLLM Leaderboard as a benchmark. Building on Qwen2.5 and Deepseek-R1, we employed techniques including supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning (RL) to enhance their financial capabilities. The fine-tuned models demonstrated substantial performance gains across a wide range of financial tasks. Moreover, we measured the data scaling law in the financial domain. Our work demonstrates the potential of large language models (LLMs) in financial applications.