HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Pooling
This work addresses the challenge of precise temporal localization in long videos for applications like video retrieval and analysis, representing a strong specific gain rather than a foundational breakthrough.
The paper tackled the problem of video temporal grounding in long, untrimmed videos by introducing HieraMamba, a hierarchical architecture that preserves temporal structure and semantic richness, achieving new state-of-the-art results on benchmarks like Ego4D-NLQ, MAD, and TACoS.
Video temporal grounding, the task of localizing the start and end times of a natural language query in untrimmed video, requires capturing both global context and fine-grained temporal detail. This challenge is particularly pronounced in long videos, where existing methods often compromise temporal fidelity by over-downsampling or relying on fixed windows. We present HieraMamba, a hierarchical architecture that preserves temporal structure and semantic richness across scales. At its core are Anchor-MambaPooling (AMP) blocks, which utilize Mamba's selective scanning to produce compact anchor tokens that summarize video content at multiple granularities. Two complementary objectives, anchor-conditioned and segment-pooled contrastive losses, encourage anchors to retain local detail while remaining globally discriminative. HieraMamba sets a new state-of-the-art on Ego4D-NLQ, MAD, and TACoS, demonstrating precise, temporally faithful localization in long, untrimmed videos.