Transfer learning optimization based on evolutionary selective fine tuning
This work addresses computational cost and overfitting issues for practitioners using deep learning in image analysis, though it is incremental as it builds on existing fine-tuning methods.
The paper tackles the problem of computational inefficiency and overfitting in transfer learning by introducing BioTune, an evolutionary adaptive fine-tuning technique that selectively fine-tunes layers, achieving competitive or improved accuracy and efficiency across nine image classification datasets.
Deep learning has shown substantial progress in image analysis. However, the computational demands of large, fully trained models remain a consideration. Transfer learning offers a strategy for adapting pre-trained models to new tasks. Traditional fine-tuning often involves updating all model parameters, which can potentially lead to overfitting and higher computational costs. This paper introduces BioTune, an evolutionary adaptive fine-tuning technique that selectively fine-tunes layers to enhance transfer learning efficiency. BioTune employs an evolutionary algorithm to identify a focused set of layers for fine-tuning, aiming to optimize model performance on a given target task. Evaluation across nine image classification datasets from various domains indicates that BioTune achieves competitive or improved accuracy and efficiency compared to existing fine-tuning methods such as AutoRGN and LoRA. By concentrating the fine-tuning process on a subset of relevant layers, BioTune reduces the number of trainable parameters, potentially leading to decreased computational cost and facilitating more efficient transfer learning across diverse data characteristics and distributions.