A Picture Tells a Thousand Words -- About You! User Interest Profiling from User Generated Visual Content
This addresses the need for improved personalization in recommender systems by leveraging visual content, but it is incremental as it extends existing text-based methods to images.
The paper tackles the problem of inferring user interests from images posted on social networks, proposing an approach that aggregates image-level content and uses similarity and category propagation, and demonstrates its effectiveness on a Pinterest dataset.
Inference of online social network users' attributes and interests has been an active research topic. Accurate identification of users' attributes and interests is crucial for improving the performance of personalization and recommender systems. Most of the existing works have focused on textual content generated by the users and have successfully used it for predicting users' interests and other identifying attributes. However, little attention has been paid to user generated visual content (images) that is becoming increasingly popular and pervasive in recent times. We posit that images posted by users on online social networks are a reflection of topics they are interested in and propose an approach to infer user attributes from images posted by them. We analyze the content of individual images and then aggregate the image-level knowledge to infer user-level interest distribution. We employ image-level similarity to propagate the label information between images, as well as utilize the image category information derived from the user created organization structure to further propagate the category-level knowledge for all images. A real life social network dataset created from Pinterest is used for evaluation and the experimental results demonstrate the effectiveness of our proposed approach.