Delta-KNN: Improving Demonstration Selection in In-Context Learning for Alzheimer's Disease Detection
This work addresses the challenge of accurate early diagnosis of Alzheimer's Disease through linguistic analysis, offering a potential tool for healthcare, though it is incremental as it builds on existing in-context learning frameworks.
The paper tackled the problem of poor performance of conventional in-context learning methods for Alzheimer's Disease detection from text by introducing Delta-KNN, a novel demonstration selection strategy, which achieved new state-of-the-art results, surpassing supervised classifiers when using the Llama-3.1 model.
Alzheimer's Disease (AD) is a progressive neurodegenerative disorder that leads to dementia, and early intervention can greatly benefit from analyzing linguistic abnormalities. In this work, we explore the potential of Large Language Models (LLMs) as health assistants for AD diagnosis from patient-generated text using in-context learning (ICL), where tasks are defined through a few input-output examples. Empirical results reveal that conventional ICL methods, such as similarity-based selection, perform poorly for AD diagnosis, likely due to the inherent complexity of this task. To address this, we introduce Delta-KNN, a novel demonstration selection strategy that enhances ICL performance. Our method leverages a delta score to assess the relative gains of each training example, coupled with a KNN-based retriever that dynamically selects optimal "representatives" for a given input. Experiments on two AD detection datasets across three open-source LLMs demonstrate that Delta-KNN consistently outperforms existing ICL baselines. Notably, when using the Llama-3.1 model, our approach achieves new state-of-the-art results, surpassing even supervised classifiers.