IIMedGPT: Promoting Large Language Model Capabilities of Medical Tasks by Efficient Human Preference Alignment
This work addresses the challenge of aligning LLMs with medical tasks for healthcare professionals, but it is incremental as it builds on existing methods like DPO.
The authors tackled the problem of aligning large language models with medical user instructions by introducing a medical instruction dataset and fine-tuning with Direct Preference Optimization, resulting in a model that outperforms existing medical models in medical dialogue.
Recent researches of large language models(LLM), which is pre-trained on massive general-purpose corpora, have achieved breakthroughs in responding human queries. However, these methods face challenges including limited data insufficiency to support extensive pre-training and can not align responses with users' instructions. To address these issues, we introduce a medical instruction dataset, CMedINS, containing six medical instructions derived from actual medical tasks, which effectively fine-tunes LLM in conjunction with other data. Subsequently, We launch our medical model, IIMedGPT, employing an efficient preference alignment method, Direct preference Optimization(DPO). The results show that our final model outperforms existing medical models in medical dialogue.Datsets, Code and model checkpoints will be released upon acceptance.