With Whom Do I Interact? Detecting Social Interactions in Egocentric Photo-streams
This addresses the challenge of understanding social behavior from egocentric data, but it is incremental as it builds on existing sociological concepts and neural network methods.
The paper tackles the problem of automatically detecting when a user wearing a low-frame-rate wearable camera engages in social interactions by analyzing captured photos, achieving promising results on a dataset of 30,000 images.
Given a user wearing a low frame rate wearable camera during a day, this work aims to automatically detect the moments when the user gets engaged into a social interaction solely by reviewing the automatically captured photos by the worn camera. The proposed method, inspired by the sociological concept of F-formation, exploits distance and orientation of the appearing individuals -with respect to the user- in the scene from a bird-view perspective. As a result, the interaction pattern over the sequence can be understood as a two-dimensional time series that corresponds to the temporal evolution of the distance and orientation features over time. A Long-Short Term Memory-based Recurrent Neural Network is then trained to classify each time series. Experimental evaluation over a dataset of 30.000 images has shown promising results on the proposed method for social interaction detection in egocentric photo-streams.