U-LanD: Uncertainty-Driven Video Landmark Detection
This work provides a method for more accurate video landmark detection for medical professionals, particularly in scenarios with limited and noisy labeled data, which is a common problem in medical imaging.
This paper introduces U-LanD, a framework for detecting key frames and landmarks in videos, specifically addressing challenges with noisy and sparse training labels. Applied to ultrasound heart videos, U-LanD significantly outperformed a non-Bayesian state-of-the-art method by an absolute margin of 42% in R2 score, using data from 4,493 patients.
This paper presents U-LanD, a framework for joint detection of key frames and landmarks in videos. We tackle a specifically challenging problem, where training labels are noisy and highly sparse. U-LanD builds upon a pivotal observation: a deep Bayesian landmark detector solely trained on key video frames, has significantly lower predictive uncertainty on those frames vs. other frames in videos. We use this observation as an unsupervised signal to automatically recognize key frames on which we detect landmarks. As a test-bed for our framework, we use ultrasound imaging videos of the heart, where sparse and noisy clinical labels are only available for a single frame in each video. Using data from 4,493 patients, we demonstrate that U-LanD can exceedingly outperform the state-of-the-art non-Bayesian counterpart by a noticeable absolute margin of 42% in R2 score, with almost no overhead imposed on the model size. Our approach is generic and can be potentially applied to other challenging data with noisy and sparse training labels.