Semantic Summarization of Egocentric Photo Stream Events
This addresses the need for efficient retrieval and summarization of large amounts of data from wearable cameras for users, but it is incremental as it builds on existing image retrieval methods with new filtering and ranking techniques.
This work tackles the problem of automatically summarizing egocentric photo streams from wearable cameras by using an image retrieval approach, achieving 95.74% expert satisfaction and a Mean Opinion Score of 4.57 out of 5.0 on a dataset of 7,110 images.
With the rapid increase of users of wearable cameras in recent years and of the amount of data they produce, there is a strong need for automatic retrieval and summarization techniques. This work addresses the problem of automatically summarizing egocentric photo streams captured through a wearable camera by taking an image retrieval perspective. After removing non-informative images by a new CNN-based filter, images are ranked by relevance to ensure semantic diversity and finally re-ranked by a novelty criterion to reduce redundancy. To assess the results, a new evaluation metric is proposed which takes into account the non-uniqueness of the solution. Experimental results applied on a database of 7,110 images from 6 different subjects and evaluated by experts gave 95.74% of experts satisfaction and a Mean Opinion Score of 4.57 out of 5.0. Source code is available at https://github.com/imatge-upc/egocentric-2017-lta