Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology
For epileptologists and neurologists, this work shows that general-purpose MLLMs can be adapted for automated seizure semiology analysis, offering interpretable diagnostic support without task-specific training.
This pilot study evaluates zero-shot MLLMs for recognizing pathological movements in seizure videos, finding they outperform fine-tuned CNN/ViT baselines on 13 of 18 features and achieve 94.3% explanation faithfulness, demonstrating potential for clinical diagnostic assistance.
Multimodal Large Language Models (MLLMs) have demonstrated robust capabilities in recognizing everyday human activities, yet their potential for analyzing clinically significant involuntary movements in neurological disorders remains largely unexplored. This pilot study evaluates the capability of MLLMs for automated recognition of pathological movements in seizure videos. We assessed the zero-shot performance of state-of-the-art MLLMs on 20 ILAE-defined semiological features across 90 clinical seizure recordings. MLLMs outperformed fine-tuned Convolutional Neural Network (CNN) and Vision Transformer (ViT) baseline models on 13 of 18 features without task-specific training, demonstrating particular strength in recognizing salient postural and contextual features while struggling with subtle, high-frequency movements. Feature-targeted signal enhancement (facial cropping, pose estimation, audio denoising) improved performance on 10 of 20 features. Expert evaluation showed that 94.3 percent of MLLM-generated explanations for correctly predicted cases achieved at least 60 percent faithfulness scores, aligning with epileptologist reasoning. These findings demonstrate the potential of adapting general-purpose MLLMs for specialized clinical video analysis through targeted preprocessing strategies, offering a path toward interpretable, efficient diagnostic assistance. Our code is publicly available at https://github.com/LinaZhangUCLA/PathMotionMLLM.