Care about you: towards large-scale human-centric visual relationship detection
This work addresses the challenge of human-object interaction detection for applications like understanding human behavior, but it is incremental as it builds on existing visual relationship detection with a new dataset and method.
The paper tackles the problem of detecting visual relationships between objects and humans by constructing a large-scale human-centric dataset (HCVRD) with nearly 10K relationship categories, and proposes a webly-supervised model to address the long-tail distribution, achieving a strong baseline performance.
Visual relationship detection aims to capture interactions between pairs of objects in images. Relationships between objects and humans represent a particularly important subset of this problem, with implications for challenges such as understanding human behaviour, and identifying affordances, amongst others. In addressing this problem we first construct a large-scale human-centric visual relationship detection dataset (HCVRD), which provides many more types of relationship annotation (nearly 10K categories) than the previous released datasets. This large label space better reflects the reality of human-object interactions, but gives rise to a long-tail distribution problem, which in turn demands a zero-shot approach to labels appearing only in the test set. This is the first time this issue has been addressed. We propose a webly-supervised approach to these problems and demonstrate that the proposed model provides a strong baseline on our HCVRD dataset.