Learning Action Hierarchies via Hybrid Geometric Diffusion
This addresses the problem of video understanding for researchers and practitioners by improving action segmentation through hierarchical modeling, though it is incremental as it builds on existing diffusion methods.
The paper tackles temporal action segmentation by proposing HybridTAS, a framework that uses a hybrid of Euclidean and hyperbolic geometries in diffusion models to exploit hierarchical action structures, achieving state-of-the-art performance on benchmark datasets like GTEA, 50Salads, and Breakfast.
Temporal action segmentation is a critical task in video understanding, where the goal is to assign action labels to each frame in a video. While recent advances leverage iterative refinement-based strategies, they fail to explicitly utilize the hierarchical nature of human actions. In this work, we propose HybridTAS - a novel framework that incorporates a hybrid of Euclidean and hyperbolic geometries into the denoising process of diffusion models to exploit the hierarchical structure of actions. Hyperbolic geometry naturally provides tree-like relationships between embeddings, enabling us to guide the action label denoising process in a coarse-to-fine manner: higher diffusion timesteps are influenced by abstract, high-level action categories (root nodes), while lower timesteps are refined using fine-grained action classes (leaf nodes). Extensive experiments on three benchmark datasets, GTEA, 50Salads, and Breakfast, demonstrate that our method achieves state-of-the-art performance, validating the effectiveness of hyperbolic-guided denoising for the temporal action segmentation task.