NEURAL: Attention-Guided Pruning for Unified Multimodal Resource-Constrained Clinical Evaluation
This addresses storage and transmission challenges in clinical workflows, enabling efficient teleradiology without performance loss, though it appears incremental as it builds on existing vision-language models for a specific domain.
The paper tackles the problem of large multimodal medical imaging data in resource-constrained clinical settings by proposing NEURAL, a framework that uses semantics-guided compression to reduce image data size by 93.4-97.7% while maintaining diagnostic performance of 0.88-0.95 AUC for pneumonia detection.
The rapid growth of multimodal medical imaging data presents significant storage and transmission challenges, particularly in resource-constrained clinical settings. We propose NEURAL, a novel framework that addresses this by using semantics-guided data compression. Our approach repurposes cross-attention scores between the image and its radiological report from a fine-tuned generative vision-language model to structurally prune chest X-rays, preserving only diagnostically critical regions. This process transforms the image into a highly compressed, graph representation. This unified graph-based representation fuses the pruned visual graph with a knowledge graph derived from the clinical report, creating a universal data structure that simplifies downstream modeling. Validated on the MIMIC-CXR and CheXpert Plus dataset for pneumonia detection, NEURAL achieves a 93.4-97.7\% reduction in image data size while maintaining a high diagnostic performance of 0.88-0.95 AUC, outperforming other baseline models that use uncompressed data. By creating a persistent, task-agnostic data asset, NEURAL resolves the trade-off between data size and clinical utility, enabling efficient workflows and teleradiology without sacrificing performance. Our NEURAL code is available at https://github.com/basiralab/NEURAL.