Parameter Efficient Fine-tuning for Domain-specific Gastrointestinal Disease Recognition
For medical image analysis practitioners, this work offers a more efficient method to adapt pretrained models to domain-specific tasks without storing multiple full model copies.
The paper addresses the problem of distribution shifts in medical image analysis by proposing parameter-efficient fine-tuning with LoRA for gastrointestinal disease recognition, achieving better results than full fine-tuning with improved parameter efficiency.
Despite recent advancements in the field of medical image analysis with the use of pretrained foundation models, the issue of distribution shifts between cross-source images largely remains adamant. To circumvent that issue, investigators generally train a separate model for each source. However, this method becomes expensive when we fully fine-tune pretrained large models for a single dataset, as we must store multiple copies of those models. Thus, in this work, we propose using a low-rank adaptation (LoRA) module for fine-tuning downstream classification tasks. LoRAs learn lightweight task-specific low-rank matrices that perturb pretrained weights to optimize those downstream tasks. For gastrointestinal tract diseases, they exhibit significantly better results than end-to-end finetuning with improved parameter efficiency. Code is available at: github.com/sanjay931/peft-gi-recognition.