Learning Scene Context Without Images
This addresses the challenge of enabling machines to anticipate objects not immediately visible, though it appears incremental as an enhancement to existing detection methods.
The paper tackles the problem of teaching machines scene contextual knowledge without using images by introducing LMOD, a transformer-based approach that learns object relationships solely from dataset labels. The method improves visual object detection algorithms by 12% on COCO and 8% on PASCAL VOC benchmarks.
Teaching machines of scene contextual knowledge would enable them to interact more effectively with the environment and to anticipate or predict objects that may not be immediately apparent in their perceptual field. In this paper, we introduce a novel transformer-based approach called $LMOD$ ( Label-based Missing Object Detection) to teach scene contextual knowledge to machines using an attention mechanism. A distinctive aspect of the proposed approach is its reliance solely on labels from image datasets to teach scene context, entirely eliminating the need for the actual image itself. We show how scene-wide relationships among different objects can be learned using a self-attention mechanism. We further show that the contextual knowledge gained from label based learning can enhance performance of other visual based object detection algorithm.