LSTC: Boosting Atomic Action Detection with Long-Short-Term Context
This work addresses atomic action detection for video analysis, offering an incremental improvement by integrating long and short-term contexts.
The paper tackles atomic action detection in videos by analyzing temporal reliance through a Long-Short Term Context (LSTC) framework, decomposing it into short-term and long-term branches that independently predict actions and merge results, achieving significant performance gains on mainstream benchmarks.
In this paper, we place the atomic action detection problem into a Long-Short Term Context (LSTC) to analyze how the temporal reliance among video signals affect the action detection results. To do this, we decompose the action recognition pipeline into short-term and long-term reliance, in terms of the hypothesis that the two kinds of context are conditionally independent given the objective action instance. Within our design, a local aggregation branch is utilized to gather dense and informative short-term cues, while a high order long-term inference branch is designed to reason the objective action class from high-order interaction between actor and other person or person pairs. Both branches independently predict the context-specific actions and the results are merged in the end. We demonstrate that both temporal grains are beneficial to atomic action recognition. On the mainstream benchmarks of atomic action detection, our design can bring significant performance gain from the existing state-of-the-art pipeline. The code of this project can be found at [this url](https://github.com/TencentYoutuResearch/ActionDetection-LSTC)