HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space
For surgical AI, this addresses the limitation of flat embeddings in capturing hierarchical surgical knowledge, improving phase recognition across procedures and institutions.
HyperVLP introduces a hyperbolic space pre-training framework for surgical video-language models that preserves hierarchical structure (actions, steps, phases), achieving consistent gains in zero- and few-shot phase recognition across multiple surgical benchmarks.
Surgical vision-language foundation models typically adopt educational materials, such as surgical lecture videos, to transfer surgical knowledge encoded in language into visual representations. These knowledge are multi-dimensional and hierarchical: fine-grained action cues appear in narration, mid-level key steps are summarized in subsection headings, and global procedural context, such as patient history and surgical strategy, is described in abstract texts. Prior work largely collapses these heterogeneous signals into a single flat embedding space, implicitly assuming independence across hierarchy levels. However, this is suboptimal because it ignores cross-level semantic containment, e.g., actions belong to steps, steps compose phases, weakens long-range dependency modeling. To this end, we propose a hyperbolic surgical video-language pre-training framework that explicitly preserves the hierarchical structure by mitigating structural false negatives induced by procedural context and enforcing semantic consistency between parent phases and their constituent child steps. Extensive experiments on multiple surgical benchmarks show consistent gains in zero- and few-shot phase recognition across procedures and institutions.