Which Emoji Talks Best for My Picture?
This work addresses the need for more expressive social media posts by improving emoji recommendation accuracy, though it is incremental as it builds on existing methods with domain knowledge.
The paper tackled the problem of recommending emojis for multimedia posts by incorporating domain knowledge from Emojinet, resulting in a 9.6% improvement over state-of-the-art methods on a Twitter dataset.
Emojis have evolved as complementary sources for expressing emotion in social-media platforms where posts are mostly composed of texts and images. In order to increase the expressiveness of the social media posts, users associate relevant emojis with their posts. Incorporating domain knowledge has improved machine understanding of text. In this paper, we investigate whether domain knowledge for emoji can improve the accuracy of emoji recommendation task in case of multimedia posts composed of image and text. Our emoji recommendation can suggest accurate emojis by exploiting both visual and textual content from social media posts as well as domain knowledge from Emojinet. Experimental results using pre-trained image classifiers and pre-trained word embedding models on Twitter dataset show that our results outperform the current state-of-the-art by 9.6\%. We also present a user study evaluation of our recommendation system on a set of images chosen from MSCOCO dataset.