Are Clinical T5 Models Better for Clinical Text?
This work addresses the effectiveness of clinical language models for healthcare applications, showing incremental benefits with limited generalization.
The study evaluated whether clinical T5 models outperform generic T5 models on clinical text tasks, finding they offer only marginal improvements and perform worse when tested on different clinical domains.
Large language models with a transformer-based encoder/decoder architecture, such as T5, have become standard platforms for supervised tasks. To bring these technologies to the clinical domain, recent work has trained new or adapted existing models to clinical data. However, the evaluation of these clinical T5 models and comparison to other models has been limited. Are the clinical T5 models better choices than FLAN-tuned generic T5 models? Do they generalize better to new clinical domains that differ from the training sets? We comprehensively evaluate these models across several clinical tasks and domains. We find that clinical T5 models provide marginal improvements over existing models, and perform worse when evaluated on different domains. Our results inform future choices in developing clinical LLMs.