Iterative Shrinking for Referring Expression Grounding Using Deep Reinforcement Learning
This work addresses the problem of localizing objects in images based on complex queries for computer vision applications, offering an incremental improvement over existing methods.
The paper tackles the proposal-free referring expression grounding task by introducing an iterative shrinking mechanism using deep reinforcement learning, which improves accuracy by 4.32% over the previous state-of-the-art on the RefCOCOg dataset.
In this paper, we are tackling the proposal-free referring expression grounding task, aiming at localizing the target object according to a query sentence, without relying on off-the-shelf object proposals. Existing proposal-free methods employ a query-image matching branch to select the highest-score point in the image feature map as the target box center, with its width and height predicted by another branch. Such methods, however, fail to utilize the contextual relation between the target and reference objects, and lack interpretability on its reasoning procedure. To solve these problems, we propose an iterative shrinking mechanism to localize the target, where the shrinking direction is decided by a reinforcement learning agent, with all contents within the current image patch comprehensively considered. Beside, the sequential shrinking process enables to demonstrate the reasoning about how to iteratively find the target. Experiments show that the proposed method boosts the accuracy by 4.32% against the previous state-of-the-art (SOTA) method on the RefCOCOg dataset, where query sentences are long and complex, with many targets referred by other reference objects.