Generating customized prompts for Zero-Shot Rare Event Medical Image Classification using LLM
This addresses the challenge of rare event detection in medicine, where data scarcity and variability hinder deep learning, by providing a privacy-preserving method that enhances classification without additional training.
The paper tackles the problem of rare event medical image classification by generating customized prompts using domain-specific expert knowledge, which are then used by large language models for zero-shot classification, outperforming state-of-the-art techniques.
Rare events, due to their infrequent occurrences, do not have much data, and hence deep learning techniques fail in estimating the distribution for such data. Open-vocabulary models represent an innovative approach to image classification. Unlike traditional models, these models classify images into any set of categories specified with natural language prompts during inference. These prompts usually comprise manually crafted templates (e.g., 'a photo of a {}') that are filled in with the names of each category. This paper introduces a simple yet effective method for generating highly accurate and contextually descriptive prompts containing discriminative characteristics. Rare event detection, especially in medicine, is more challenging due to low inter-class and high intra-class variability. To address these, we propose a novel approach that uses domain-specific expert knowledge on rare events to generate customized and contextually relevant prompts, which are then used by large language models for image classification. Our zero-shot, privacy-preserving method enhances rare event classification without additional training, outperforming state-of-the-art techniques.