Domain-Adapted Small Language Models for Reliable Clinical Triage
For emergency departments, this work demonstrates that privacy-preserving, institution-specific SLMs can provide reliable decision support for clinical triage, addressing mistriage and workflow inefficiencies.
This study shows that fine-tuned small language models (SLMs), particularly Qwen2.5-7B, can outperform proprietary large language models like GPT-4o in Emergency Severity Index (ESI) triage, reducing discordance and clinically significant errors through domain adaptation.
Accurate and consistent Emergency Severity Index (ESI) assignment remains a persistent challenge in emergency departments, where highly variable free-text triage documentation contributes to mistriage and workflow inefficiencies. This study evaluates whether open-source small language models (SLMs) can serve as reliable, privacy-preserving decision-support tools for clinical triage. We systematically compared multiple SLMs across diverse prompting pipelines and found that clinical vignettes, concise summaries of triage narratives, yielded the most accurate predictions. The SLM, Qwen2.5-7B, demonstrated the strongest balance of accuracy, stability, and computational efficiency. Through large-scale domain adaptation using expert-curated and silver-standard pediatric triage data, fine-tuned Qwen2.5-7B models substantially reduced discordance and clinically significant errors, outperforming all baseline SLMs and advanced proprietary large language models (LLMs, e.g., GPT-4o). These findings highlight the feasibility of institution-specific SLMs for reliable, privacy-preserving ESI decision support and underscore the importance of targeted fine-tuning over more complex inference strategies.