Visual Relationship Detection Based on Guided Proposals and Semantic Knowledge Distillation
This work addresses the challenge of understanding diverse object interactions in real-world images for computer vision applications, representing an incremental improvement.
The paper tackled the problem of detecting visual relationships between objects in images by incorporating semantic knowledge and relevance estimation, achieving a 68.5% relative gain in recall at 100 and a 32.7% gain from knowledge distillation on the Visual Genome dataset.
A thorough comprehension of image content demands a complex grasp of the interactions that may occur in the natural world. One of the key issues is to describe the visual relationships between objects. When dealing with real world data, capturing these very diverse interactions is a difficult problem. It can be alleviated by incorporating common sense in a network. For this, we propose a framework that makes use of semantic knowledge and estimates the relevance of object pairs during both training and test phases. Extracted from precomputed models and training annotations, this information is distilled into the neural network dedicated to this task. Using this approach, we observe a significant improvement on all classes of Visual Genome, a challenging visual relationship dataset. A 68.5% relative gain on the recall at 100 is directly related to the relevance estimate and a 32.7% gain to the knowledge distillation.