Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology
This addresses performance gaps in computational pathology tasks like sub-typing and diagnosis by enabling fine-tuning within downstream MIL models, though it is incremental as it builds on existing MIL frameworks.
The paper tackles the limitation of multiple instance learning (MIL) in computational pathology, where offline feature extractors restrict adaptability, by proposing a Re-embedded Regional Transformer (R^2T) for online feature re-embedding, which improves ResNet-50-based MIL models to foundation model-level performance and further enhances foundation model features, with R^2T-MIL outperforming other latest methods by a large margin.
Multiple instance learning (MIL) is the most widely used framework in computational pathology, encompassing sub-typing, diagnosis, prognosis, and more. However, the existing MIL paradigm typically requires an offline instance feature extractor, such as a pre-trained ResNet or a foundation model. This approach lacks the capability for feature fine-tuning within the specific downstream tasks, limiting its adaptability and performance. To address this issue, we propose a Re-embedded Regional Transformer (R$^2$T) for re-embedding the instance features online, which captures fine-grained local features and establishes connections across different regions. Unlike existing works that focus on pre-training powerful feature extractor or designing sophisticated instance aggregator, R$^2$T is tailored to re-embed instance features online. It serves as a portable module that can seamlessly integrate into mainstream MIL models. Extensive experimental results on common computational pathology tasks validate that: 1) feature re-embedding improves the performance of MIL models based on ResNet-50 features to the level of foundation model features, and further enhances the performance of foundation model features; 2) the R$^2$T can introduce more significant performance improvements to various MIL models; 3) R$^2$T-MIL, as an R$^2$T-enhanced AB-MIL, outperforms other latest methods by a large margin.The code is available at: https://github.com/DearCaat/RRT-MIL.