The Impact of Skin Tone Label Granularity on the Performance and Fairness of AI Based Dermatology Image Classification Models
This addresses bias in dermatology AI models for patients with diverse skin tones, but it is incremental as it builds on existing critiques of the FST scale.
The study investigated how the granularity of skin tone labels in the Fitzpatrick Skin Tone (FST) scale affects AI models for classifying benign vs. malignant skin lesions, finding that models trained on FST-specific data with three groups (FST 1/2, 3/4, 5/6) generally performed better than a general model, and reducing granularity (e.g., from 1/2 and 3/4 to 1/2/3/4) harmed performance.
Artificial intelligence (AI) models to automatically classify skin lesions from dermatology images have shown promising performance but also susceptibility to bias by skin tone. The most common way of representing skin tone information is the Fitzpatrick Skin Tone (FST) scale. The FST scale has been criticised for having greater granularity in its skin tone categories for lighter-skinned subjects. This paper conducts an investigation of the impact (on performance and bias) on AI classification models of granularity in the FST scale. By training multiple AI models to classify benign vs. malignant lesions using FST-specific data of differing granularity, we show that: (i) when training models using FST-specific data based on three groups (FST 1/2, 3/4 and 5/6), performance is generally better for models trained on FST-specific data compared to a general model trained on FST-balanced data; (ii) reducing the granularity of FST scale information (from 1/2 and 3/4 to 1/2/3/4) can have a detrimental effect on performance. Our results highlight the importance of the granularity of FST groups when training lesion classification models. Given the question marks over possible human biases in the choice of categories in the FST scale, this paper provides evidence for a move away from the FST scale in fair AI research and a transition to an alternative scale that better represents the diversity of human skin tones.