ZePT: Zero-Shot Pan-Tumor Segmentation via Query-Disentangling and Self-Prompting
This addresses the long-tailed distribution issue in medical image analysis for clinicians and researchers, enabling segmentation of unseen tumor types, though it appears incremental as it builds on existing query-based methods.
The paper tackles the problem of segmenting rare or novel tumor categories not seen during training in medical images by proposing ZePT, a zero-shot pan-tumor segmentation framework that uses query-disentangling and self-prompting, achieving performance superiority over previous methods in various tumor segmentation tasks.
The long-tailed distribution problem in medical image analysis reflects a high prevalence of common conditions and a low prevalence of rare ones, which poses a significant challenge in developing a unified model capable of identifying rare or novel tumor categories not encountered during training. In this paper, we propose a new zero-shot pan-tumor segmentation framework (ZePT) based on query-disentangling and self-prompting to segment unseen tumor categories beyond the training set. ZePT disentangles the object queries into two subsets and trains them in two stages. Initially, it learns a set of fundamental queries for organ segmentation through an object-aware feature grouping strategy, which gathers organ-level visual features. Subsequently, it refines the other set of advanced queries that focus on the auto-generated visual prompts for unseen tumor segmentation. Moreover, we introduce query-knowledge alignment at the feature level to enhance each query's discriminative representation and generalizability. Extensive experiments on various tumor segmentation tasks demonstrate the performance superiority of ZePT, which surpasses the previous counterparts and evidence the promising ability for zero-shot tumor segmentation in real-world settings.