Do Better ImageNet Models Transfer Better... for Image Recommendation?
This work addresses a practical problem for researchers and practitioners in image recommendation by clarifying model selection and fine-tuning strategies, though it is incremental as it builds on existing transfer learning methods.
The study investigated whether better-performing ImageNet models transfer more effectively to content-based image recommendation tasks, finding that higher ImageNet accuracy does not consistently lead to better transfer performance (e.g., NASNet vs. ResNet), and that fine-tuning can help with small datasets but its effectiveness varies.
Visual embeddings from Convolutional Neural Networks (CNN) trained on the ImageNet dataset for the ILSVRC challenge have shown consistently good performance for transfer learning and are widely used in several tasks, including image recommendation. However, some important questions have not yet been answered in order to use these embeddings for a larger scope of recommendation domains: a) Do CNNs that perform better in ImageNet are also better for transfer learning in content-based image recommendation?, b) Does fine-tuning help to improve performance? and c) Which is the best way to perform the fine-tuning? In this paper we compare several CNN models pre-trained with ImageNet to evaluate their transfer learning performance to an artwork image recommendation task. Our results indicate that models with better performance in the ImageNet challenge do not always imply better transfer learning for recommendation tasks (e.g. NASNet vs. ResNet). Our results also show that fine-tuning can be helpful even with a small dataset, but not every fine-tuning works. Our results can inform other researchers and practitioners on how to train their CNNs for better transfer learning towards image recommendation systems.