Anticipating human actions by correlating past with the future with Jaccard similarity measures
This work addresses the problem of predicting human actions from partial observations for applications like surveillance and robotics, representing an incremental improvement with specific gains.
The paper tackles early action recognition and anticipation by correlating past and future features using novel Jaccard similarity measures, achieving state-of-the-art results with accuracies of 91.7% on UCF101 and 83.5% on JHMDB for early recognition, and 20.35 and 41.8 top-1 accuracy on Epic-Kitchen55 and Breakfast datasets for anticipation.
We propose a framework for early action recognition and anticipation by correlating past features with the future using three novel similarity measures called Jaccard vector similarity, Jaccard cross-correlation and Jaccard Frobenius inner product over covariances. Using these combinations of novel losses and using our framework, we obtain state-of-the-art results for early action recognition in UCF101 and JHMDB datasets by obtaining 91.7 % and 83.5 % accuracy respectively for an observation percentage of 20. Similarly, we obtain state-of-the-art results for Epic-Kitchen55 and Breakfast datasets for action anticipation by obtaining 20.35 and 41.8 top-1 accuracy respectively.