Training-Free Adaptation of New-Generation LLMs using Legacy Clinical Models
This work enables resource-constrained healthcare institutions to leverage state-of-the-art general-domain LLMs without costly retraining, by reusing existing clinical models.
The authors propose Cross-Architecture Proxy Tuning (CAPT), a training-free method to adapt new general-domain LLMs using older clinical models, achieving average improvements of +17.6% over UniTE and +41.4% over proxy tuning on six clinical tasks.
Adapting language models to the clinical domain through continued pretraining and instruction tuning requires costly retraining for each new model generation. We propose Cross-Architecture Proxy Tuning (CAPT), a model-ensembling approach that enables training-free adaptation of state-of-the-art general-domain models using existing clinical models. CAPT supports models with disjoint vocabularies, leveraging contrastive decoding to selectively inject clinically relevant signals while preserving the general-domain model's reasoning and fluency. On six clinical classification and text-generation tasks, CAPT with a new-generation general-domain model and an older-generation clinical model consistently outperforms both models individually and state-of-the-art ensembling approaches (average +17.6\% over UniTE, +41.4\% over proxy tuning across tasks). Through token-level analysis and physician case studies, we demonstrate that CAPT amplifies clinically actionable language, reduces context errors, and increases clinical specificity. This technique especially benefits healthcare institutions with constrained computational capacity that cannot support iterative clinical training and want to adopt emerging general-domain model advances.