Fully-Coupled Two-Stream Spatiotemporal Networks for Extremely Low Resolution Action Recognition
This addresses privacy concerns in smart devices by enabling reliable action recognition with minimal visual information, though it is incremental as it builds on existing two-stream approaches.
The paper tackles the problem of human action recognition on extremely low resolution videos (e.g., 12x16 pixels) to protect privacy, proposing a fully-coupled two-stream spatiotemporal architecture that achieves significant improvements over state-of-the-art methods on two datasets.
A major emerging challenge is how to protect people's privacy as cameras and computer vision are increasingly integrated into our daily lives, including in smart devices inside homes. A potential solution is to capture and record just the minimum amount of information needed to perform a task of interest. In this paper, we propose a fully-coupled two-stream spatiotemporal architecture for reliable human action recognition on extremely low resolution (e.g., 12x16 pixel) videos. We provide an efficient method to extract spatial and temporal features and to aggregate them into a robust feature representation for an entire action video sequence. We also consider how to incorporate high resolution videos during training in order to build better low resolution action recognition models. We evaluate on two publicly-available datasets, showing significant improvements over the state-of-the-art.