CPARR: Category-based Proposal Analysis for Referring Relationships
This addresses the task of referring relationships in computer vision, which is incremental as it builds on existing methods like SSAS.
The paper tackles the problem of localizing subject and object entities in images based on relationship queries, introducing a proposal-based method that achieves state-of-the-art performance on Visual Relationship Detection and Visual Genome datasets.
The task of referring relationships is to localize subject and object entities in an image satisfying a relationship query, which is given in the form of \texttt{<subject, predicate, object>}. This requires simultaneous localization of the subject and object entities in a specified relationship. We introduce a simple yet effective proposal-based method for referring relationships. Different from the existing methods such as SSAS, our method can generate a high-resolution result while reducing its complexity and ambiguity. Our method is composed of two modules: a category-based proposal generation module to select the proposals related to the entities and a predicate analysis module to score the compatibility of pairs of selected proposals. We show state-of-the-art performance on the referring relationship task on two public datasets: Visual Relationship Detection and Visual Genome.