Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Pathology Analysis
This work addresses inefficiencies in pathology diagnosis using whole slide images, offering an interactive, clinically aligned approach for auxiliary diagnosis, though it is incremental as it builds on existing LVLM frameworks.
The paper tackled the problem of limited input resolution in large vision-language models for pathology image analysis by proposing mixed task-guided feature enhancement and prompt-guided detail feature completion, resulting in the OmniPath model that significantly outperforms existing methods in diagnostic accuracy and efficiency on a dataset of 490K samples.
Pathological diagnosis is vital for determining disease characteristics, guiding treatment, and assessing prognosis, relying heavily on detailed, multi-scale analysis of high-resolution whole slide images (WSI). However, existing large vision-language models (LVLMs) are limited by input resolution constraints, hindering their efficiency and accuracy in pathology image analysis. To overcome these issues, we propose two innovative strategies: the mixed task-guided feature enhancement, which directs feature extraction toward lesion-related details across scales, and the prompt-guided detail feature completion, which integrates coarse- and fine-grained features from WSI based on specific prompts without compromising inference speed. Leveraging a comprehensive dataset of 490K samples from diverse pathology tasks, we trained the pathology-specialized LVLM, OmniPath. Extensive experiments demonstrate that this model significantly outperforms existing methods in diagnostic accuracy and efficiency, providing an interactive, clinically aligned approach for auxiliary diagnosis in a wide range of pathology applications.