OutdoorSent: Sentiment Analysis of Urban Outdoor Images by Using Semantic and Deep Features
This work addresses sentiment analysis for urban planning and social science applications, but it is incremental as it builds on existing methods with minor enhancements.
The authors tackled sentiment analysis of outdoor urban images by comparing ConvNet architectures and incorporating semantic scene attributes, finding that semantic knowledge improved accuracy across all models and that focusing on outdoor images was more effective than including indoor ones.
Opinion mining in outdoor images posted by users during different activities can provide valuable information to better understand urban areas. In this regard, we propose a framework to classify the sentiment of outdoor images shared by users on social networks. We compare the performance of state-of-the-art ConvNet architectures, and one specifically designed for sentiment analysis. We also evaluate how the merging of deep features and semantic information derived from the scene attributes can improve classification and cross-dataset generalization performance. The evaluation explores a novel dataset, namely OutdoorSent, and other datasets publicly available. We observe that the incorporation of knowledge about semantic attributes improves the accuracy of all ConvNet architectures studied. Besides, we found that exploring only images related to the context of the study, outdoor in our case, is recommended, i.e., indoor images were not significantly helpful. Furthermore, we demonstrated the applicability of our results in the city of Chicago, USA, showing that they can help to improve the knowledge of subjective characteristics of different areas of the city. For instance, particular areas of the city tend to concentrate more images of a specific class of sentiment, which are also correlated with median income, opening up opportunities in different fields.