Bridging the gap between Natural and Medical Images through Deep Colorization
This work addresses data scarcity in medical imaging by focusing on color adaptation, offering a practical solution for diagnostic tasks, though it appears incremental as it builds on existing transfer learning approaches.
The paper tackles the challenge of medical image diagnosis with limited data by proposing a dedicated network module for color adaptation, which when combined with transfer learning of classification backbones achieves efficient diagnostic recognition on X-ray images, particularly under data scarcity conditions.
Deep learning has thrived by training on large-scale datasets. However, in many applications, as for medical image diagnosis, getting massive amount of data is still prohibitive due to privacy, lack of acquisition homogeneity and annotation cost. In this scenario, transfer learning from natural image collections is a standard practice that attempts to tackle shape, texture and color discrepancies all at once through pretrained model fine-tuning. In this work, we propose to disentangle those challenges and design a dedicated network module that focuses on color adaptation. We combine learning from scratch of the color module with transfer learning of different classification backbones, obtaining an end-to-end, easy-to-train architecture for diagnostic image recognition on X-ray images. Extensive experiments showed how our approach is particularly efficient in case of data scarcity and provides a new path for further transferring the learned color information across multiple medical datasets.