Anticipating Daily Intention using On-Wrist Motion Triggered Sensing
This work addresses the need for efficient intention anticipation in daily life applications, such as guiding users to power outlets, but it is incremental as it builds on existing sensing and neural network methods.
The paper tackles the problem of anticipating human intentions from actions using an on-wrist motion-triggered sensing system, achieving accuracies of 92.68%, 90.85%, and 97.56% for three users while processing only 29% of visual observations on average.
Anticipating human intention by observing one's actions has many applications. For instance, picking up a cellphone, then a charger (actions) implies that one wants to charge the cellphone (intention). By anticipating the intention, an intelligent system can guide the user to the closest power outlet. We propose an on-wrist motion triggered sensing system for anticipating daily intentions, where the on-wrist sensors help us to persistently observe one's actions. The core of the system is a novel Recurrent Neural Network (RNN) and Policy Network (PN), where the RNN encodes visual and motion observation to anticipate intention, and the PN parsimoniously triggers the process of visual observation to reduce computation requirement. We jointly trained the whole network using policy gradient and cross-entropy loss. To evaluate, we collect the first daily "intention" dataset consisting of 2379 videos with 34 intentions and 164 unique action sequences. Our method achieves 92.68%, 90.85%, 97.56% accuracy on three users while processing only 29% of the visual observation on average.