No Modality Left Behind: Dynamic Model Generation for Incomplete Medical Data
This provides an efficient solution for real-world clinical environments where multi-modal medical data is often partially incomplete, though it is incremental as it builds on hypernetwork techniques for a specific domain bottleneck.
The paper tackles the problem of training deep learning models on multi-modal medical imaging data with incomplete samples by proposing a hypernetwork-based method that dynamically generates task-specific models conditioned on available modalities, resulting in an absolute accuracy increase of up to 8% compared to state-of-the-art methods when trained on data with 25% completeness.
In real world clinical environments, training and applying deep learning models on multi-modal medical imaging data often struggles with partially incomplete data. Standard approaches either discard missing samples, require imputation or repurpose dropout learning schemes, limiting robustness and generalizability. To address this, we propose a hypernetwork-based method that dynamically generates task-specific classification models conditioned on the set of available modalities. Instead of training a fixed model, a hypernetwork learns to predict the parameters of a task model adapted to available modalities, enabling training and inference on all samples, regardless of completeness. We compare this approach with (1) models trained only on complete data, (2) state of the art channel dropout methods, and (3) an imputation-based method, using artificially incomplete datasets to systematically analyze robustness to missing modalities. Results demonstrate superior adaptability of our method, outperforming state of the art approaches with an absolute increase in accuracy of up to 8% when trained on a dataset with 25% completeness (75% of training data with missing modalities). By enabling a single model to generalize across all modality configurations, our approach provides an efficient solution for real-world multi-modal medical data analysis.