Enhancing Reasoning Skills in Small Persian Medical Language Models Can Outperform Large-Scale Data Training
This addresses the challenge of developing effective medical AI tools in underrepresented languages like Persian with limited data, though it is incremental as it builds on existing methods like RLAIF and DPO.
The study tackled the problem of enhancing reasoning skills in small Persian language models for medical question answering by using RLAIF and DPO to train on a translated dataset, resulting in a model that outperformed a larger predecessor trained on 57 million tokens with a smaller dataset of 4.5 million tokens.
Enhancing reasoning capabilities in small language models is critical for specialized applications such as medical question answering, particularly in underrepresented languages like Persian. In this study, we employ Reinforcement Learning with AI Feedback (RLAIF) and Direct preference optimization (DPO) to improve the reasoning skills of a general-purpose Persian language model. To achieve this, we translated a multiple-choice medical question-answering dataset into Persian and used RLAIF to generate rejected-preferred answer pairs, which are essential for DPO training. By prompting both teacher and student models to produce Chain-of-Thought (CoT) reasoning responses, we compiled a dataset containing correct and incorrect reasoning trajectories. This dataset, comprising 2 million tokens in preferred answers and 2.5 million tokens in rejected ones, was used to train a baseline model, significantly enhancing its medical reasoning capabilities in Persian. Remarkably, the resulting model outperformed its predecessor, gaokerena-V, which was trained on approximately 57 million tokens, despite leveraging a much smaller dataset. These results highlight the efficiency and effectiveness of reasoning-focused training approaches in developing domain-specific language models with limited data availability.