Fluoroformer: Scaling multiple instance learning to multiplexed images via attention-based channel fusion
This work addresses the adaptation of AI techniques to emerging spatial biology assays for computational pathology, representing an incremental advancement.
The paper tackled the problem of applying multiple instance learning to multiplexed whole slide images in computational pathology, introducing the Fluoroformer module that uses attention-based channel fusion and achieved strong prognostic performance on 434 non-small cell lung cancer samples.
Though multiple instance learning (MIL) has been a foundational strategy in computational pathology for processing whole slide images (WSIs), current approaches are designed for traditional hematoxylin and eosin (H&E) slides rather than emerging multiplexed technologies. Here, we present an MIL strategy, the Fluoroformer module, that is specifically tailored to multiplexed WSIs by leveraging scaled dot-product attention (SDPA) to interpretably fuse information across disparate channels. On a cohort of 434 non-small cell lung cancer (NSCLC) samples, we show that the Fluoroformer both obtains strong prognostic performance and recapitulates immuno-oncological hallmarks of NSCLC. Our technique thereby provides a path for adapting state-of-the-art AI techniques to emerging spatial biology assays.