Visual-Attribute Prompt Learning for Progressive Mild Cognitive Impairment Prediction
This work addresses the challenge of early and accurate diagnosis of progressive cognitive decline in patients, representing an incremental improvement by integrating multi-modal data with a novel prompt learning approach.
The paper tackles the problem of predicting progressive Mild Cognitive Impairment (pMCI) by proposing a transformer-based network that fuses brain imaging and clinical data using prompt fine-tuning, achieving superior performance compared to state-of-the-art methods and outperforming fully fine-tuning baselines in knowledge transfer from Alzheimer's Disease prediction.
Deep learning (DL) has been used in the automatic diagnosis of Mild Cognitive Impairment (MCI) and Alzheimer's Disease (AD) with brain imaging data. However, previous methods have not fully exploited the relation between brain image and clinical information that is widely adopted by experts in practice. To exploit the heterogeneous features from imaging and tabular data simultaneously, we propose the Visual-Attribute Prompt Learning-based Transformer (VAP-Former), a transformer-based network that efficiently extracts and fuses the multi-modal features with prompt fine-tuning. Furthermore, we propose a Prompt fine-Tuning (PT) scheme to transfer the knowledge from AD prediction task for progressive MCI (pMCI) diagnosis. In details, we first pre-train the VAP-Former without prompts on the AD diagnosis task and then fine-tune the model on the pMCI detection task with PT, which only needs to optimize a small amount of parameters while keeping the backbone frozen. Next, we propose a novel global prompt token for the visual prompts to provide global guidance to the multi-modal representations. Extensive experiments not only show the superiority of our method compared with the state-of-the-art methods in pMCI prediction but also demonstrate that the global prompt can make the prompt learning process more effective and stable. Interestingly, the proposed prompt learning model even outperforms the fully fine-tuning baseline on transferring the knowledge from AD to pMCI.