PinView: Implicit Feedback in Content-Based Image Retrieval
This work addresses the challenge of enhancing image retrieval accuracy and user experience for users of content-based systems, representing an incremental improvement through novel feedback integration methods.
The paper tackles the problem of improving content-based image retrieval by incorporating implicit user feedback, such as eye movements or clicks, to learn a user-specific similarity metric and balance exploration-exploitation during searches, resulting in PinView outperforming the original PicSOM system and achieving the best results with combined implicit and explicit feedback in online user experiments.
This paper describes PinView, a content-based image retrieval system that exploits implicit relevance feedback collected during a search session. PinView contains several novel methods to infer the intent of the user. From relevance feedback, such as eye movements or pointer clicks, and visual features of images, PinView learns a similarity metric between images which depends on the current interests of the user. It then retrieves images with a specialized online learning algorithm that balances the tradeoff between exploring new images and exploiting the already inferred interests of the user. We have integrated PinView to the content-based image retrieval system PicSOM, which enables applying PinView to real-world image databases. With the new algorithms PinView outperforms the original PicSOM, and in online experiments with real users the combination of implicit and explicit feedback gives the best results.