Pay Attention with Focus: A Novel Learning Scheme for Classification of Whole Slide Images
This work addresses the problem of WSI classification for medical imaging, but it is incremental as it builds on existing patch-based and attention methods.
The authors tackled the challenge of classifying whole slide images (WSIs) with deep learning by proposing a two-stage approach that extracts and encodes patches, then uses an attention-weighted averaging scheme for diagnosis prediction, achieving robust and effective results.
Deep learning methods such as convolutional neural networks (CNNs) are difficult to directly utilize to analyze whole slide images (WSIs) due to the large image dimensions. We overcome this limitation by proposing a novel two-stage approach. First, we extract a set of representative patches (called mosaic) from a WSI. Each patch of a mosaic is encoded to a feature vector using a deep network. The feature extractor model is fine-tuned using hierarchical target labels of WSIs, i.e., anatomic site and primary diagnosis. In the second stage, a set of encoded patch-level features from a WSI is used to compute the primary diagnosis probability through the proposed Pay Attention with Focus scheme, an attention-weighted averaging of predicted probabilities for all patches of a mosaic modulated by a trainable focal factor. Experimental results show that the proposed model can be robust, and effective for the classification of WSIs.