FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024
This work addresses the problem of contextual alignment for financial market applications, offering a domain-specific solution that is incremental in nature.
The study tackled the challenge of generating precise trading signals for financial markets by fine-tuning a large language model (LLaMA-3.2-3B-Instruct) with a novel Reinforcement Learning from Market Feedback (RLMF) prompt framework, resulting in outperformance of baseline methods in signal consistency and trading outcomes, as evidenced by winning Task II of the FinRL Contest 2024.
In response to Task II of the FinRL Challenge at ACM ICAIF 2024, this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Market Feedback (RLMF). Our framework incorporates market-specific features and short-term price dynamics to generate more precise trading signals. Traditional LLMs, while competent in sentiment analysis, lack contextual alignment for financial market applications. To bridge this gap, we fine-tune the LLaMA-3.2-3B-Instruct model using a custom RLMF prompt design that integrates historical market data and reward-based feedback. Our evaluation shows that this RLMF-tuned framework outperforms baseline methods in signal consistency and achieving tighter trading outcomes; awarded as winner of Task II. You can find the code for this project on GitHub.