OptiBox: Breaking the Limits of Proposals for Visual Grounding
This work addresses a key bottleneck in visual grounding for image-lingual reasoning tasks, offering a method that reduces data requirements while maintaining performance.
The authors tackled the limitation of bounding box proposal quality in visual grounding by proposing OptiBox, a progressive query-guided refinement architecture, achieving state-of-the-art performance on the Flickr30k Entities dataset and competitive results with only 3% of training data.
The problem of language grounding has attracted much attention in recent years due to its pivotal role in more general image-lingual high level reasoning tasks (e.g., image captioning, VQA). Despite the tremendous progress in visual grounding, the performance of most approaches has been hindered by the quality of bounding box proposals obtained in the early stages of all recent pipelines. To address this limitation, we propose a general progressive query-guided bounding box refinement architecture (OptiBox) that leverages global image encoding for added context. We apply this architecture in the context of the GroundeR model, first introduced in 2016, which has a number of unique and appealing properties, such as the ability to learn in the semi-supervised setting by leveraging cyclic language-reconstruction. Using GroundeR + OptiBox and a simple semantic language reconstruction loss that we propose, we achieve state-of-the-art grounding performance in the supervised setting on Flickr30k Entities dataset. More importantly, we are able to surpass many recent fully supervised models with only 50% of training data and perform competitively with as low as 3%.