Automated Disentangling Analysis of Skin Colour for Lesion Images
This addresses a domain-specific issue for dermatology and machine learning applications by improving model robustness to color variations in skin images.
The paper tackled the problem of degraded performance in skin lesion classification due to mismatched skin color captured in images (SCCI) between training and deployment, by proposing a disentangling framework that enables faithful counterfactual editing and achieves competitive classification performance.
Machine-learning models working on skin images often have degraded performance when the skin colour captured in images (SCCI) differs between training and deployment. Such differences arise from entangled environmental factors (e.g., illumination, camera settings), and intrinsic factors (e.g., skin tone) that cannot be accurately described by a single "skin tone" scalar. To mitigate such colour mismatch, we propose a skin-colour disentangling framework that adapts disentanglement-by-compression to learn a structured, manipulable latent space for SCCI from unlabelled dermatology images. To prevent information leakage that hinders proper learning of dark colour features, we introduce a randomized, mostly monotonic decolourization mapping. To suppress unintended colour shifts of localized patterns (e.g., ink marks, scars) during colour manipulation, we further propose a geometry-aligned post-processing step. Together, these components enable faithful counterfactual editing and answering an essential question: "What would this skin condition look like under a different SCCI?", as well as direct colour transfer between images and controlled traversal along physically meaningful directions (e.g., blood perfusion, camera white balance), enabling educational visualization of skin conditions under varying SCCI. We demonstrate that dataset-level augmentation and colour normalization based on our framework achieve competitive lesion classification performance.