Structured Visual Search via Composition-aware Learning
It addresses visual search for structured queries, offering an incremental improvement in efficiency and performance for computer vision applications.
The paper tackles visual search with structured queries by leveraging continuous relationships in object compositions through symmetry and equivariance, resulting in a more efficient technique that achieves considerable performance gains on MS-COCO and HICO-DET benchmarks.
This paper studies visual search using structured queries. The structure is in the form of a 2D composition that encodes the position and the category of the objects. The transformation of the position and the category of the objects leads to a continuous-valued relationship between visual compositions, which carries highly beneficial information, although not leveraged by previous techniques. To that end, in this work, our goal is to leverage these continuous relationships by using the notion of symmetry in equivariance. Our model output is trained to change symmetrically with respect to the input transformations, leading to a sensitive feature space. Doing so leads to a highly efficient search technique, as our approach learns from fewer data using a smaller feature space. Experiments on two large-scale benchmarks of MS-COCO and HICO-DET demonstrates that our approach leads to a considerable gain in the performance against competing techniques.