Cross Modality 3D Navigation Using Reinforcement Learning and Neural Style Transfer
This work addresses the challenge of user guidance during diagnostic imaging acquisition to improve consistency and case comparison in medical domains, representing an incremental advance by integrating existing techniques for a specific application.
The paper tackles the problem of 3D navigation in medical imaging by using Multi-Agent Reinforcement Learning (MARL) and Neural Style Transfer to create synthetic CT environments, enabling agents to generalize to clinical CT volumes without labelled data and condition on 2D slices for applications like ultrasound imaging.
This paper presents the use of Multi-Agent Reinforcement Learning (MARL) to perform navigation in 3D anatomical volumes from medical imaging. We utilize Neural Style Transfer to create synthetic Computed Tomography (CT) agent gym environments and assess the generalization capabilities of our agents to clinical CT volumes. Our framework does not require any labelled clinical data and integrates easily with several image translation techniques, enabling cross modality applications. Further, we solely condition our agents on 2D slices, breaking grounds for 3D guidance in much more difficult imaging modalities, such as ultrasound imaging. This is an important step towards user guidance during the acquisition of standardised diagnostic view planes, improving diagnostic consistency and facilitating better case comparison.