Efficacy of MRI data harmonization in the age of machine learning. A multicenter study across 36 datasets
This work addresses data leakage issues in multicenter MRI studies for researchers using machine learning, representing an incremental improvement in harmonization methods.
The study tackled the problem of data leakage in MRI data harmonization for machine learning by proposing a harmonizer transformer and efficacy measurement, demonstrating that their approach removed site effects and prevented data leakage in age prediction from MRI data across 36 datasets.
Pooling publicly-available MRI data from multiple sites allows to assemble extensive groups of subjects, increase statistical power, and promote data reuse with machine learning techniques. The harmonization of multicenter data is necessary to reduce the confounding effect associated with non-biological sources of variability in the data. However, when applied to the entire dataset before machine learning, the harmonization leads to data leakage, because information outside the training set may affect model building, and potentially falsely overestimate performance. We propose a 1) measurement of the efficacy of data harmonization; 2) harmonizer transformer, i.e., an implementation of the ComBat harmonization allowing its encapsulation among the preprocessing steps of a machine learning pipeline, avoiding data leakage. We tested these tools using brain T1-weighted MRI data from 1740 healthy subjects acquired at 36 sites. After harmonization, the site effect was removed or reduced, and we showed the data leakage effect in predicting individual age from MRI data, highlighting that introducing the harmonizer transformer into a machine learning pipeline allows for avoiding data leakage.