What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factors
This work addresses the uncertainty in applying transfer learning to medical imaging, which is crucial for improving diagnostic tools in healthcare.
The study investigated factors influencing the effectiveness of transfer learning from ImageNet to medical imaging, finding that it is generally beneficial and highlighting the significant role of feature reuse in its success.
Transfer learning is a standard technique to transfer knowledge from one domain to another. For applications in medical imaging, transfer from ImageNet has become the de-facto approach, despite differences in the tasks and image characteristics between the domains. However, it is unclear what factors determine whether - and to what extent - transfer learning to the medical domain is useful. The long-standing assumption that features from the source domain get reused has recently been called into question. Through a series of experiments on several medical image benchmark datasets, we explore the relationship between transfer learning, data size, the capacity and inductive bias of the model, as well as the distance between the source and target domain. Our findings suggest that transfer learning is beneficial in most cases, and we characterize the important role feature reuse plays in its success.