IVOct 31, 2023
Enhanced Synthetic MRI Generation from CT Scans Using CycleGAN with Feature ExtractionSaba Nikbakhsh, Lachin Naghashyar, Morteza Valizadeh et al.
In the field of radiotherapy, accurate imaging and image registration are of utmost importance for precise treatment planning. Magnetic Resonance Imaging (MRI) offers detailed imaging without being invasive and excels in soft-tissue contrast, making it a preferred modality for radiotherapy planning. However, the high cost of MRI, longer acquisition time, and certain health considerations for patients pose challenges. Conversely, Computed Tomography (CT) scans offer a quicker and less expensive imaging solution. To bridge these modalities and address multimodal alignment challenges, we introduce an approach for enhanced monomodal registration using synthetic MRI images. Utilizing unpaired data, this paper proposes a novel method to produce these synthetic MRI images from CT scans, leveraging CycleGANs and feature extractors. By building upon the foundational work on Cycle-Consistent Adversarial Networks and incorporating advancements from related literature, our methodology shows promising results, outperforming several state-of-the-art methods. The efficacy of our approach is validated by multiple comparison metrics.
CVFeb 6
Towards Understanding Multimodal Fine-Tuning: Spatial FeaturesLachin Naghashyar, Hunar Batra, Ashkan Khakzar et al.
Contemporary Vision-Language Models (VLMs) achieve strong performance on a wide range of tasks by pairing a vision encoder with a pre-trained language model, fine-tuned for visual-text inputs. Yet despite these gains, it remains unclear how language backbone representations adapt during multimodal training and when vision-specific capabilities emerge. In this work, we present the first mechanistic analysis of VLM adaptation. Using stage-wise model diffing, a technique that isolates representational changes introduced during multimodal fine-tuning, we reveal how a language model learns to "see". We first identify vision-preferring features that emerge or reorient during fine-tuning. We then show that a selective subset of these features reliably encodes spatial relations, revealed through controlled shifts to spatial prompts. Finally, we trace the causal activation of these features to a small group of attention heads. Our findings show that stage-wise model diffing reveals when and where spatially grounded multimodal features arise. It also provides a clearer view of modality fusion by showing how visual grounding reshapes features that were previously text-only. This methodology enhances the interpretability of multimodal training and provides a foundation for understanding and refining how pretrained language models acquire vision-grounded capabilities.
IVJun 29, 2025
Improving Myocardial Infarction Detection via Synthetic ECG PretrainingLachin Naghashyar
Myocardial infarction is a major cause of death globally, and accurate early diagnosis from electrocardiograms (ECGs) remains a clinical priority. Deep learning models have shown promise for automated ECG interpretation, but require large amounts of labeled data, which are often scarce in practice. We propose a physiology-aware pipeline that (i) synthesizes 12-lead ECGs with tunable MI morphology and realistic noise, and (ii) pre-trains recurrent and transformer classifiers with self-supervised masked-autoencoding plus a joint reconstruction-classification objective. We validate the realism of synthetic ECGs via statistical and visual analysis, confirming that key morphological features are preserved. Pretraining on synthetic data consistently improved classification performance, particularly in low-data settings, with AUC gains of up to 4 percentage points. These results show that controlled synthetic ECGs can help improve MI detection when real clinical data is limited.