Improving Object Detection in Art Images Using Only Style Transfer
This work addresses the challenge of detecting objects like people in art images, which is important for applications in art analysis and digital humanities, but it is incremental as it builds on existing style transfer and detection methods.
The paper tackled the cross-depiction problem in object detection for art images by fine-tuning a Faster R-CNN network on a dataset generated via AdaIn style transfer from COCO, achieving a significant improvement in state-of-the-art performance on the People-Art dataset.
Despite recent advances in object detection using deep learning neural networks, these neural networks still struggle to identify objects in art images such as paintings and drawings. This challenge is known as the cross depiction problem and it stems in part from the tendency of neural networks to prioritize identification of an object's texture over its shape. In this paper we propose and evaluate a process for training neural networks to localize objects - specifically people - in art images. We generate a large dataset for training and validation by modifying the images in the COCO dataset using AdaIn style transfer. This dataset is used to fine-tune a Faster R-CNN object detection network, which is then tested on the existing People-Art testing dataset. The result is a significant improvement on the state of the art and a new way forward for creating datasets to train neural networks to process art images.