HookNet: multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images
This work addresses the problem of accurate tissue type prediction in histopathology images for medical diagnosis, particularly in cancer contexts, by improving segmentation through better contextual integration, though it is incremental as it builds on existing encoder-decoder architectures.
The authors tackled semantic segmentation in histopathology whole-slide images by proposing HookNet, a multi-resolution convolutional neural network that combines context and details via a hooking mechanism, achieving superior performance compared to single-resolution U-Net models and a recent multi-resolution model in tasks like multi-class tissue segmentation in breast cancer and segmentation of tertiary lymphoid structures in lung cancer.
We propose HookNet, a semantic segmentation model for histopathology whole-slide images, which combines context and details via multiple branches of encoder-decoder convolutional neural networks. Concentricpatches at multiple resolutions with different fields of view are used to feed different branches of HookNet, and intermediate representations are combined via a hooking mechanism. We describe a framework to design and train HookNet for achieving high-resolution semantic segmentation and introduce constraints to guarantee pixel-wise alignment in feature maps during hooking. We show the advantages of using HookNet in two histopathology image segmentation tasks where tissue type prediction accuracy strongly depends on contextual information, namely (1) multi-class tissue segmentation in breast cancer and, (2) segmentation of tertiary lymphoid structures and germinal centers in lung cancer. Weshow the superiority of HookNet when compared with single-resolution U-Net models working at different resolutions as well as with a recently published multi-resolution model for histopathology image segmentation