Batch-Based Activity Recognition from Egocentric Photo-Streams
This work addresses a technical challenge in assistive technology like health monitoring, but it is incremental as it builds on existing deep learning methods for temporal modeling.
The paper tackles activity recognition from low-frame-rate egocentric photo-streams by proposing a batch-driven deep learning approach using LSTM units to capture temporal evolution of high-level features, with experimental results on a public dataset from three users demonstrating its validity.
Activity recognition from long unstructured egocentric photo-streams has several applications in assistive technology such as health monitoring and frailty detection, just to name a few. However, one of its main technical challenges is to deal with the low frame rate of wearable photo-cameras, which causes abrupt appearance changes between consecutive frames. In consequence, important discriminatory low-level features from motion such as optical flow cannot be estimated. In this paper, we present a batch-driven approach for training a deep learning architecture that strongly rely on Long short-term units to tackle this problem. We propose two different implementations of the same approach that process a photo-stream sequence using batches of fixed size with the goal of capturing the temporal evolution of high-level features. The main difference between these implementations is that one explicitly models consecutive batches by overlapping them. Experimental results over a public dataset acquired by three users demonstrate the validity of the proposed architectures to exploit the temporal evolution of convolutional features over time without relying on event boundaries.