T. Gade

Ningcheng Li, Jonathan Wakim, Yilun Koethe et al.

Purpose: To evaluate manual and automatic registration times as well as accuracy with augmented reality during alignment of a holographic 3-dimensional (3D) model onto the real-world environment. Method: 18 participants in various stages of clinical training across two academic centers registered a 3D CT phantom model onto a CT grid using the HoloLens 2 augmented reality headset 3 consecutive times. Registration times and accuracy were compared among different registration methods (hand gesture, Xbox controller, and automatic registration), levels of clinical experience, and consecutive attempts. Registration times were also compared with prior HoloLens 1 data. Results: Mean aggregate manual registration times were 27.7, 24.3, and 72.8 seconds for one-handed gesture, two-handed gesture, and Xbox controller, respectively; mean automatic registration time was 5.3s (ANOVA p<0.0001). No significant difference in registration times was found among attendings, residents and fellows, and medical students (p>0.05). Significant improvements in registration times were detected across consecutive attempts using hand gestures (p<0.01). Compared with previously reported HoloLens 1 experience, hand gesture registration times were 81.7% faster (p<0.05). Registration accuracies were not significantly different across manual registration methods, measuring at 5.9, 9.5, and 8.6 mm with one-handed gesture, two-handed gesture, and Xbox controller, respectively (p>0.05). Conclusions: Manual registration times decreased significantly with updated hand gesture maneuvers on HoloLens 2 versus HoloLens 1, approaching the registration times of automatic registration and outperforming Xbox controller mediated registration. These results will encourage wider clinical integration of HoloLens 2 in procedural medical care.

Brian J. Park, Stephen J. Hunt, Gregory J. Nadolski et al.

Background: Out-of-plane lesions pose challenges for CT-guided interventions. Augmented reality (AR) headset devices have evolved and are readily capable to provide virtual 3D guidance to improve CT-guided targeting. Purpose: To describe the design of a three-dimensional (3D) AR-assisted navigation system using HoloLens 2 and evaluate its performance through CT-guided simulations. Materials and Methods: A prospective trial was performed assessing CT-guided needle targeting on an abdominal phantom with and without AR guidance. A total of 8 operators with varying clinical experience were enrolled and performed a total of 86 needle passes. Procedure efficiency, radiation dose, and complication rates were compared with and without AR guidance. Vector analysis of the first needle pass was also performed. Results: Average total number of needle passes to reach the target reduced from 7.4 passes without AR to 3.4 passes with AR (54.2% decrease, p=0.011). Average dose-length product (DLP) decreased from 538 mGy-cm without AR to 318 mGy-cm with AR (41.0% decrease, p=0.009). Complication rate of hitting a non-targeted lesion decreased from 11.9% without AR (7/59 needle passes) to 0% with AR (0/27 needle passes). First needle passes were more nearly aligned with the ideal target trajectory with AR versus without AR (4.6° vs 8.0° offset, respectively, p=0.018). Medical students, residents, and attendings all performed at the same level with AR guidance. Conclusions: 3D AR guidance can provide significant improvements in procedural efficiency and radiation dose savings for targeting challenging, out-of-plane lesions. AR guidance elevated the performance of all operators to the same level irrespective of prior clinical experience.

Andong Cao, Ali Dhanaliwala, Jianbo Shi et al.

Augmented reality has the potential to improve operating room workflow by allowing physicians to "see" inside a patient through the projection of imaging directly onto the surgical field. For this to be useful the acquired imaging must be quickly and accurately registered with patient and the registration must be maintained. Here we describe a method for projecting a CT scan with Microsoft Hololens and then aligning that projection to a set of fiduciary markers. Radio-opaque stickers with unique QR-codes are placed on an object prior to acquiring a CT scan. The location of the markers in the CT scan are extracted and the CT scan is converted into a 3D surface object. The 3D object is then projected using the Hololens onto a table on which the same markers are placed. We designed an algorithm that aligns the markers on the 3D object with the markers on the table. To extract the markers and convert the CT into a 3D object took less than 5 seconds. To align three markers, it took $0.9 \pm 0.2$ seconds to achieve an accuracy of $5 \pm 2$ mm. These findings show that it is feasible to use a combined radio-opaque optical marker, placed on a patient prior to a CT scan, to subsequently align the acquired CT scan with the patient.