Distilling Knowledge from Object Classification to Aesthetics Assessment
This work improves image aesthetics assessment for applications like photo editing and social media, but it is incremental as it builds on existing distillation techniques.
The paper tackles the challenge of image aesthetics assessment (IAA) by addressing the abstract nature of aesthetic labels, which makes it hard to relate diverse image contents to the same labels. It proposes distilling knowledge from pre-trained object classification models into an IAA model, resulting in a 4.8% improvement in SRCC and outperforming 10 previous methods.
In this work, we point out that the major dilemma of image aesthetics assessment (IAA) comes from the abstract nature of aesthetic labels. That is, a vast variety of distinct contents can correspond to the same aesthetic label. On the one hand, during inference, the IAA model is required to relate various distinct contents to the same aesthetic label. On the other hand, when training, it would be hard for the IAA model to learn to distinguish different contents merely with the supervision from aesthetic labels, since aesthetic labels are not directly related to any specific content. To deal with this dilemma, we propose to distill knowledge on semantic patterns for a vast variety of image contents from multiple pre-trained object classification (POC) models to an IAA model. Expecting the combination of multiple POC models can provide sufficient knowledge on various image contents, the IAA model can easier learn to relate various distinct contents to a limited number of aesthetic labels. By supervising an end-to-end single-backbone IAA model with the distilled knowledge, the performance of the IAA model is significantly improved by 4.8% in SRCC compared to the version trained only with ground-truth aesthetic labels. On specific categories of images, the SRCC improvement brought by the proposed method can achieve up to 7.2%. Peer comparison also shows that our method outperforms 10 previous IAA methods.