Semantic Image Cropping
This work addresses the need for more relevant image cropping for specific entities, though it appears incremental as it builds upon existing aesthetic cropping methods by adding semantic considerations.
The paper tackles the problem of image cropping by introducing semantics as an additional dimension beyond aesthetics, showing that existing aesthetic-only cropping algorithms perform poorly on semantic cropping tasks. The authors provide a new dataset and a deep learning system that incorporates semantic information, demonstrating improved performance on their semantic cropping benchmark.
Automatic image cropping techniques are commonly used to enhance the aesthetic quality of an image; they do it by detecting the most beautiful or the most salient parts of the image and removing the unwanted content to have a smaller image that is more visually pleasing. In this thesis, I introduce an additional dimension to the problem of cropping, semantics. I argue that image cropping can also enhance the image's relevancy for a given entity by using the semantic information contained in the image. I call this problem, Semantic Image Cropping. To support my argument, I provide a new dataset containing 100 images with at least two different entities per image and four ground truth croppings collected using Amazon Mechanical Turk. I use this dataset to show that state-of-the-art cropping algorithms that only take into account aesthetics do not perform well in the problem of semantic image cropping. Additionally, I provide a new deep learning system that takes not just aesthetics but also semantics into account to generate image croppings, and I evaluate its performance using my new semantic cropping dataset, showing that using the semantic information of an image can help to produce better croppings.