SEMBED: Semantic Embedding of Egocentric Action Videos
This work addresses the challenge of handling ambiguous verb labels in egocentric video analysis, which is incremental as it builds on existing embedding and classification techniques.
The authors tackled the problem of interpreting egocentric object interaction videos with ambiguous verb annotations by embedding them in a semantic-visual graph to estimate label probabilities, resulting in a method that outperforms SVM classification by over 5% on a dataset of 1225 videos.
We present SEMBED, an approach for embedding an egocentric object interaction video in a semantic-visual graph to estimate the probability distribution over its potential semantic labels. When object interactions are annotated using unbounded choice of verbs, we embrace the wealth and ambiguity of these labels by capturing the semantic relationships as well as the visual similarities over motion and appearance features. We show how SEMBED can interpret a challenging dataset of 1225 freely annotated egocentric videos, outperforming SVM classification by more than 5%.