SR-Clustering: Semantic Regularized Clustering for Egocentric Photo Streams Segmentation
This addresses the tedious process of locating relevant information in large unstructured collections of egocentric images for users of wearable cameras, representing an incremental improvement.
The paper tackles the problem of organizing egocentric photo streams into meaningful segments by extracting contextual and semantic information using CNNs and language processing, and grouping images based on shared attributes and temporal coherence. Experiments on nearly 17,000 images show it outperforms state-of-the-art methods.
While wearable cameras are becoming increasingly popular, locating relevant information in large unstructured collections of egocentric images is still a tedious and time consuming processes. This paper addresses the problem of organizing egocentric photo streams acquired by a wearable camera into semantically meaningful segments. First, contextual and semantic information is extracted for each image by employing a Convolutional Neural Networks approach. Later, by integrating language processing, a vocabulary of concepts is defined in a semantic space. Finally, by exploiting the temporal coherence in photo streams, images which share contextual and semantic attributes are grouped together. The resulting temporal segmentation is particularly suited for further analysis, ranging from activity and event recognition to semantic indexing and summarization. Experiments over egocentric sets of nearly 17,000 images, show that the proposed approach outperforms state-of-the-art methods.