ActionSwitch: Class-agnostic Detection of Simultaneous Actions in Streaming Videos
This addresses a common but overlooked challenge in online temporal action localization for streaming video applications, offering broader applicability in scenarios with overlapping actions or unavailable class labels.
The paper tackles the problem of detecting overlapping actions in streaming videos without relying on class information, introducing ActionSwitch, which achieves state-of-the-art performance on complex datasets like Epic-Kitchens 100 and FineAction.
Online Temporal Action Localization (On-TAL) is a critical task that aims to instantaneously identify action instances in untrimmed streaming videos as soon as an action concludes -- a major leap from frame-based Online Action Detection (OAD). Yet, the challenge of detecting overlapping actions is often overlooked even though it is a common scenario in streaming videos. Current methods that can address concurrent actions depend heavily on class information, limiting their flexibility. This paper introduces ActionSwitch, the first class-agnostic On-TAL framework capable of detecting overlapping actions. By obviating the reliance on class information, ActionSwitch provides wider applicability to various situations, including overlapping actions of the same class or scenarios where class information is unavailable. This approach is complemented by the proposed "conservativeness loss", which directly embeds a conservative decision-making principle into the loss function for On-TAL. Our ActionSwitch achieves state-of-the-art performance in complex datasets, including Epic-Kitchens 100 targeting the challenging egocentric view and FineAction consisting of fine-grained actions.