Microscope 2.0: An Augmented Reality Microscope with Real-time Artificial Intelligence Integration
This addresses the problem of diagnostic variability and limited access to pathologists in cancer histopathology, offering a cost-effective solution to improve accuracy and efficiency, though it is incremental in applying existing AI methods to a new hardware integration.
The paper tackles the challenge of integrating AI into real-world cancer diagnosis workflows by proposing an Augmented Reality Microscope (ARM) that overlays AI-based information in real-time, demonstrating its utility in detecting lymph node metastases in breast cancer and identifying prostate cancer with low latency.
The brightfield microscope is instrumental in the visual examination of both biological and physical samples at sub-millimeter scales. One key clinical application has been in cancer histopathology, where the microscopic assessment of the tissue samples is used for the diagnosis and staging of cancer and thus guides clinical therapy. However, the interpretation of these samples is inherently subjective, resulting in significant diagnostic variability. Moreover, in many regions of the world, access to pathologists is severely limited due to lack of trained personnel. In this regard, Artificial Intelligence (AI) based tools promise to improve the access and quality of healthcare. However, despite significant advances in AI research, integration of these tools into real-world cancer diagnosis workflows remains challenging because of the costs of image digitization and difficulties in deploying AI solutions. Here we propose a cost-effective solution to the integration of AI: the Augmented Reality Microscope (ARM). The ARM overlays AI-based information onto the current view of the sample through the optical pathway in real-time, enabling seamless integration of AI into the regular microscopy workflow. We demonstrate the utility of ARM in the detection of lymph node metastases in breast cancer and the identification of prostate cancer with a latency that supports real-time workflows. We anticipate that ARM will remove barriers towards the use of AI in microscopic analysis and thus improve the accuracy and efficiency of cancer diagnosis. This approach is applicable to other microscopy tasks and AI algorithms in the life sciences and beyond.