Segmentor-Guided Counterfactual Fine-Tuning for Locally Coherent and Targeted Image Synthesis
This addresses the need for locally coherent and targeted image synthesis in medical imaging, such as for data augmentation or disease modeling, though it is incremental as it builds on existing counterfactual methods.
The paper tackled the problem of generating structure-specific counterfactual images without requiring tedious pixel-level label maps, and demonstrated realistic chest radiographs and promising results for modeling coronary artery disease.
Counterfactual image generation is a powerful tool for augmenting training data, de-biasing datasets, and modeling disease. Current approaches rely on external classifiers or regressors to increase the effectiveness of subject-level interventions (e.g., changing the patient's age). For structure-specific interventions (e.g., changing the area of the left lung in a chest radiograph), we show that this is insufficient, and can result in undesirable global effects across the image domain. Previous work used pixel-level label maps as guidance, requiring a user to provide hypothetical segmentations which are tedious and difficult to obtain. We propose Segmentor-guided Counterfactual Fine-Tuning (Seg-CFT), which preserves the simplicity of intervening on scalar-valued, structure-specific variables while producing locally coherent and effective counterfactuals. We demonstrate the capability of generating realistic chest radiographs, and we show promising results for modeling coronary artery disease. Code: https://github.com/biomedia-mira/seg-cft.