HOLa: HoloLens Object Labeling
This work addresses the need for efficient annotation in medical AR, particularly for open liver surgery and phantom experiments, though it is incremental as it builds on existing SAM-Track algorithms.
The paper tackles the challenge of object tracking in medical AR applications by introducing HOLa, a HoloLens-Object-Labeling application that automates single object annotation, increasing labeling speed by over 500 times while achieving Dice scores of 0.875 to 0.982, comparable to human performance.
In the context of medical Augmented Reality (AR) applications, object tracking is a key challenge and requires a significant amount of annotation masks. As segmentation foundation models like the Segment Anything Model (SAM) begin to emerge, zero-shot segmentation requires only minimal human participation obtaining high-quality object masks. We introduce a HoloLens-Object-Labeling (HOLa) Unity and Python application based on the SAM-Track algorithm that offers fully automatic single object annotation for HoloLens 2 while requiring minimal human participation. HOLa does not have to be adjusted to a specific image appearance and could thus alleviate AR research in any application field. We evaluate HOLa for different degrees of image complexity in open liver surgery and in medical phantom experiments. Using HOLa for image annotation can increase the labeling speed by more than 500 times while providing Dice scores between 0.875 and 0.982, which are comparable to human annotators. Our code is publicly available at: https://github.com/mschwimmbeck/HOLa