Learning Effective Visual Relationship Detector on 1 GPU
This work addresses the problem of visual relationship detection in complex scenes for computer vision researchers, offering a hardware-efficient method that is incremental in optimizing existing techniques.
The paper tackled the Open Images 2019 Visual Relationship challenge by developing a three-stage solution that trains on a single GPU in under two days, achieving first place out of over 200 teams with a 5% lead over the second-place team.
We present our winning solution to the Open Images 2019 Visual Relationship challenge. This is the largest challenge of its kind to date with nearly 9 million training images. Challenge task consists of detecting objects and identifying relationships between them in complex scenes. Our solution has three stages, first object detection model is fine-tuned for the challenge classes using a novel weight transfer approach. Then, spatio-semantic and visual relationship models are trained on candidate object pairs. Finally, features and model predictions are combined to generate the final relationship prediction. Throughout the challenge we focused on minimizing the hardware requirements of our architecture. Specifically, our weight transfer approach enables much faster optimization, allowing the entire architecture to be trained on a single GPU in under two days. In addition to efficient optimization, our approach also achieves superior accuracy winning first place out of over 200 teams, and outperforming the second place team by over $5\%$ on the held-out private leaderboard.