Histo-fetch -- On-the-fly processing of gigapixel whole slide images simplifies and speeds neural network training
This addresses a bottleneck for researchers in computational pathology by providing an incremental improvement in data handling efficiency.
The authors tackled the problem of inefficient data preparation for training neural networks on gigapixel whole slide images by developing a custom pipeline called histo-fetch, which extracts random patches on-the-fly, simplifying and speeding up training without pre-processing steps.
We created a custom pipeline (histo-fetch) to efficiently extract random patches and labels from pathology whole slide images (WSIs) for input to a neural network on-the-fly. We prefetch these patches as needed during network training, avoiding the need for WSI preparation such as chopping/tiling. We demonstrate the utility of this pipeline to perform artificial stain transfer and image generation using the popular networks CycleGAN and ProGAN, respectively.