Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models
This addresses robustness and interpretability issues in medical AI for healthcare applications, though it is an incremental approach building on existing concept bottleneck and vision-language methods.
The authors tackled the problems of spurious correlations and lack of interpretability in medical image classifiers by proposing a concept bottleneck model using GPT-4 and vision-language models, achieving substantial performance improvements on datasets with confounding factors.
Medical image classification is a critical problem for healthcare, with the potential to alleviate the workload of doctors and facilitate diagnoses of patients. However, two challenges arise when deploying deep learning models to real-world healthcare applications. First, neural models tend to learn spurious correlations instead of desired features, which could fall short when generalizing to new domains (e.g., patients with different ages). Second, these black-box models lack interpretability. When making diagnostic predictions, it is important to understand why a model makes a decision for trustworthy and safety considerations. In this paper, to address these two limitations, we propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model. We systematically evaluate our method on eight medical image classification datasets to verify its effectiveness. On challenging datasets with strong confounding factors, our method can mitigate spurious correlations thus substantially outperform standard visual encoders and other baselines. Finally, we show how classification with a small number of concepts brings a level of interpretability for understanding model decisions through case studies in real medical data.