Learning Ultrasound Rendering from Cross-Sectional Model Slices for Simulated Training
This work addresses the need for high-quality, interactive ultrasound simulation for medical training, though it appears incremental as it builds on existing generative adversarial frameworks with architectural improvements.
The authors tackled the problem of generating realistic ultrasound images for virtual reality training by bypassing computationally expensive ray-tracing simulations at interactive time, instead learning to translate cross-sectional model slices to simulated frames offline. Their method achieved comparable or superior results to state-of-the-art approaches using only tissue maps as input, as demonstrated through quality metrics and an ablation study.
Purpose. Given the high level of expertise required for navigation and interpretation of ultrasound images, computational simulations can facilitate the training of such skills in virtual reality. With ray-tracing based simulations, realistic ultrasound images can be generated. However, due to computational constraints for interactivity, image quality typically needs to be compromised. Methods. We propose herein to bypass any rendering and simulation process at interactive time, by conducting such simulations during a non-time-critical offline stage and then learning image translation from cross-sectional model slices to such simulated frames. We use a generative adversarial framework with a dedicated generator architecture and input feeding scheme, which both substantially improve image quality without increase in network parameters. Integral attenuation maps derived from cross-sectional model slices, texture-friendly strided convolutions, providing stochastic noise and input maps to intermediate layers in order to preserve locality are all shown herein to greatly facilitate such translation task. Results. Given several quality metrics, the proposed method with only tissue maps as input is shown to provide comparable or superior results to a state-of-the-art that uses additional images of low-quality ultrasound renderings. An extensive ablation study shows the need and benefits from the individual contributions utilized in this work, based on qualitative examples and quantitative ultrasound similarity metrics. To that end, a local histogram statistics based error metric is proposed and demonstrated for visualization of local dissimilarities between ultrasound images.