Semantic Image Retrieval by Uniting Deep Neural Networks and Cognitive Architectures
This addresses the challenge of semantic image retrieval for applications requiring complex queries, though it is incremental as it combines existing deep learning and cognitive architecture methods.
The paper tackled the problem of semantic image retrieval by bridging the symbolic/subsymbolic gap, proposing a hybrid solution using YOLOv2 for object detection and OpenCog for query execution, which enabled retrieval of video frames with specified object classes and spatial arrangements.
Image and video retrieval by their semantic content has been an important and challenging task for years, because it ultimately requires bridging the symbolic/subsymbolic gap. Recent successes in deep learning enabled detection of objects belonging to many classes greatly outperforming traditional computer vision techniques. However, deep learning solutions capable of executing retrieval queries are still not available. We propose a hybrid solution consisting of a deep neural network for object detection and a cognitive architecture for query execution. Specifically, we use YOLOv2 and OpenCog. Queries allowing the retrieval of video frames containing objects of specified classes and specified spatial arrangement are implemented.