Extraction of Key-frames of Endoscopic Videos by using Depth Information
This work addresses the need to filter out low-quality frames in endoscopic videos for clinical diagnosis, but it appears incremental as it builds on existing depth estimation methods.
The paper tackles the problem of selecting informative frames from endoscopic videos by proposing a deep learning-based monocular depth estimation technique, which uses depth information to identify key frames and localize polyps, though no concrete performance numbers are provided.
A deep learning-based monocular depth estimation (MDE) technique is proposed for selection of most informative frames (key frames) of an endoscopic video. In most of the cases, ground truth depth maps of polyps are not readily available and that is why the transfer learning approach is adopted in our method. An endoscopic modalities generally capture thousands of frames. In this scenario, it is quite important to discard low-quality and clinically irrelevant frames of an endoscopic video while the most informative frames should be retained for clinical diagnosis. In this view, a key-frame selection strategy is proposed by utilizing the depth information of polyps. In our method, image moment, edge magnitude, and key-points are considered for adaptively selecting the key frames. One important application of our proposed method could be the 3D reconstruction of polyps with the help of extracted key frames. Also, polyps are localized with the help of extracted depth maps.