Overcoming Barriers to Data Sharing with Medical Image Generation: A Comprehensive Evaluation
This research provides valuable guidelines for medical researchers and institutions on the practical conditions under which synthetic medical images can serve as a privacy-preserving alternative to sharing real patient data, addressing a critical barrier to collaborative medical research.
This paper explores the use of Generative Adversarial Networks (GANs) to create synthetic medical imaging datasets for chest radiographs and brain CT scans, aiming to overcome privacy barriers in data sharing. The study evaluates the quality of synthetic data by comparing the performance of predictive models trained on synthetic versus real data, finding that synthetic data performance benefits from fewer unique label combinations and that label overfitting can dominate GAN training at low sample numbers per class. A reader study showed radiologists could not distinguish synthetic from real images at intermediate resolutions, but their accuracy increased at higher resolutions.
Privacy concerns around sharing personally identifiable information are a major practical barrier to data sharing in medical research. However, in many cases, researchers have no interest in a particular individual's information but rather aim to derive insights at the level of cohorts. Here, we utilize Generative Adversarial Networks (GANs) to create derived medical imaging datasets consisting entirely of synthetic patient data. The synthetic images ideally have, in aggregate, similar statistical properties to those of a source dataset but do not contain sensitive personal information. We assess the quality of synthetic data generated by two GAN models for chest radiographs with 14 different radiology findings and brain computed tomography (CT) scans with six types of intracranial hemorrhages. We measure the synthetic image quality by the performance difference of predictive models trained on either the synthetic or the real dataset. We find that synthetic data performance disproportionately benefits from a reduced number of unique label combinations. Our open-source benchmark also indicates that at low number of samples per class, label overfitting effects start to dominate GAN training. We additionally conducted a reader study in which trained radiologists do not perform better than random on discriminating between synthetic and real medical images for intermediate levels of resolutions. In accordance with our benchmark results, the classification accuracy of radiologists increases at higher spatial resolution levels. Our study offers valuable guidelines and outlines practical conditions under which insights derived from synthetic medical images are similar to those that would have been derived from real imaging data. Our results indicate that synthetic data sharing may be an attractive and privacy-preserving alternative to sharing real patient-level data in the right settings.