CST: Calibration Side-Tuning for Parameter and Memory Efficient Transfer Learning
This work addresses parameter and memory efficiency for transfer learning in object detection, which is incremental as it adapts existing transformer techniques to ResNet.
The paper tackles the challenge of efficiently coordinating multiple object detection tasks under resource constraints by introducing Calibration Side-Tuning, a lightweight fine-tuning strategy that integrates adapter and side tuning for ResNet, achieving state-of-the-art performance on five benchmark datasets with a better balance between complexity and performance.
Achieving a universally high accuracy in object detection is quite challenging, and the mainstream focus in the industry currently lies on detecting specific classes of objects. However, deploying one or multiple object detection networks requires a certain amount of GPU memory for training and storage capacity for inference. This presents challenges in terms of how to effectively coordinate multiple object detection tasks under resource-constrained conditions. This paper introduces a lightweight fine-tuning strategy called Calibration side tuning, which integrates aspects of adapter tuning and side tuning to adapt the successful techniques employed in transformers for use with ResNet. The Calibration side tuning architecture that incorporates maximal transition calibration, utilizing a small number of additional parameters to enhance network performance while maintaining a smooth training process. Furthermore, this paper has conducted an analysis on multiple fine-tuning strategies and have implemented their application within ResNet, thereby expanding the research on fine-tuning strategies for object detection networks. Besides, this paper carried out extensive experiments using five benchmark datasets. The experimental results demonstrated that this method outperforms other compared state-of-the-art techniques, and a better balance between the complexity and performance of the finetune schemes is achieved.