More From Less: Self-Supervised Knowledge Distillation for Routine Histopathology Data
This addresses the challenge of leveraging expensive imaging data for better diagnosis in histopathology, though it is incremental as it builds on existing knowledge distillation and self-supervised learning techniques.
The paper tackles the problem of limited access to high-quality medical imaging data by proposing a self-supervised knowledge distillation method that transfers insights from information-dense data to models using routine, information-sparse H&E stains, improving downstream classification accuracy to be comparable with fully-supervised baselines.
Medical imaging technologies are generating increasingly large amounts of high-quality, information-dense data. Despite the progress, practical use of advanced imaging technologies for research and diagnosis remains limited by cost and availability, so information-sparse data such as H&E stains are relied on in practice. The study of diseased tissue requires methods which can leverage these information-dense data to extract more value from routine, information-sparse data. Using self-supervised deep learning, we demonstrate that it is possible to distil knowledge during training from information-dense data into models which only require information-sparse data for inference. This improves downstream classification accuracy on information-sparse data, making it comparable with the fully-supervised baseline. We find substantial effects on the learned representations, and this training process identifies subtle features which otherwise go undetected. This approach enables the design of models which require only routine images, but contain insights from state-of-the-art data, allowing better use of the available resources.