Fine-Tuning LLMs on Small Medical Datasets: Text Classification and Normalization Effectiveness on Cardiology reports and Discharge records
This work addresses the challenge of efficiently extracting structured data from unstructured medical text for healthcare applications, but it is incremental as it applies existing fine-tuning methods to new medical data.
The study tackled the problem of automating clinical workflows by fine-tuning large language models on small medical datasets for text classification and named entity recognition, achieving comparable results to larger models with as few as 200-300 training examples.
We investigate the effectiveness of fine-tuning large language models (LLMs) on small medical datasets for text classification and named entity recognition tasks. Using a German cardiology report dataset and the i2b2 Smoking Challenge dataset, we demonstrate that fine-tuning small LLMs locally on limited training data can improve performance achieving comparable results to larger models. Our experiments show that fine-tuning improves performance on both tasks, with notable gains observed with as few as 200-300 training examples. Overall, the study highlights the potential of task-specific fine-tuning of LLMs for automating clinical workflows and efficiently extracting structured data from unstructured medical text.