Probabilistic Temporal Masked Attention for Cross-view Online Action Detection
This addresses a generalization issue in video analysis for computer vision applications, but it appears incremental as it builds on existing attention and probabilistic modeling techniques.
The paper tackles the problem of online action detection in videos being sensitive to varying viewpoints, which limits generalization to unseen sources, and proposes a Probabilistic Temporal Masked Attention model that achieves state-of-the-art performance on datasets like DAHLIA, IKEA ASM, and Breakfast under cross-view evaluation protocols.
As a critical task in video sequence classification within computer vision, Online Action Detection (OAD) has garnered significant attention. The sensitivity of mainstream OAD models to varying video viewpoints often hampers their generalization when confronted with unseen sources. To address this limitation, we propose a novel Probabilistic Temporal Masked Attention (PTMA) model, which leverages probabilistic modeling to derive latent compressed representations of video frames in a cross-view setting. The PTMA model incorporates a GRU-based temporal masked attention (TMA) cell, which leverages these representations to effectively query the input video sequence, thereby enhancing information interaction and facilitating autoregressive frame-level video analysis. Additionally, multi-view information can be integrated into the probabilistic modeling to facilitate the extraction of view-invariant features. Experiments conducted under three evaluation protocols: cross-subject (cs), cross-view (cv), and cross-subject-view (csv) show that PTMA achieves state-of-the-art performance on the DAHLIA, IKEA ASM, and Breakfast datasets.