Can Post-Training Turn LLMs into Good Medical Coders? An Empirical Study of Generative ICD Coding
For healthcare NLP practitioners, this work demonstrates that generative LLMs can be effective for ICD coding with appropriate post-training, challenging the prevailing view that discriminative models are necessary.
The study shows that post-training (supervised fine-tuning and reinforcement learning) significantly improves generative LLMs for ICD coding, achieving strong recall on full code sets, contrary to prior claims that LLMs are weak at this task. SFT provides the main capability jump, GRPO further improves code-set prediction, and PHI yields targeted macro-level gains.
Automated International Classification of Diseases (ICD) coding is a core medical-coding task for billing, epidemiology, and clinical decision support. Generative large language models (LLMs) are often reported as weak medical coders, but this finding mainly comes from inference-time settings such as prompting, retrieval, reranking, or tool use, leaving the role of task-specific post-training underexplored. We present a controlled empirical study of post-training for generative ICD coding, comparing discriminative baselines with LLM coders across prompting, supervised fine-tuning, and reinforcement learning under a common protocol and metric set. To our knowledge, this is the first study to evaluate RL-based post-training for generative LLM coders in ICD coding. We further introduce PHI, a diagnostic curriculum that extends GRPO to refine missed-code cases. Our results show that prompting-only evaluation substantially underestimates the potential of LLMs for ICD coding. SFT provides the main capability jump, GRPO further improves code-set prediction beyond SFT, and PHI provides targeted gains on macro-level performance. These findings suggest that the main bottleneck is not the generative formulation alone, but how the model is adapted and optimized for full-taxonomy recall. We release our code, data splits, and checkpoints at https://github.com/AlexandreWANG915/LLM4ICD.