Predicting Human Activities Using Stochastic Grammar
This work addresses the challenge of human activity prediction, which is important for applications like robotics and surveillance, but it appears incremental as it builds on existing grammar-based methods.
The paper tackles the problem of predicting future human activities from partially observed RGB-D videos by using a stochastic grammar model to capture compositional event structures, achieving effectiveness in semantic event parsing and activity prediction as shown in extensive experiments.
This paper presents a novel method to predict future human activities from partially observed RGB-D videos. Human activity prediction is generally difficult due to its non-Markovian property and the rich context between human and environments. We use a stochastic grammar model to capture the compositional structure of events, integrating human actions, objects, and their affordances. We represent the event by a spatial-temporal And-Or graph (ST-AOG). The ST-AOG is composed of a temporal stochastic grammar defined on sub-activities, and spatial graphs representing sub-activities that consist of human actions, objects, and their affordances. Future sub-activities are predicted using the temporal grammar and Earley parsing algorithm. The corresponding action, object, and affordance labels are then inferred accordingly. Extensive experiments are conducted to show the effectiveness of our model on both semantic event parsing and future activity prediction.