Neural Message Passing for Visual Relationship Detection
This work addresses the challenge of detecting interactions between objects in images for computer vision applications, representing an incremental improvement by integrating graph-based methods with existing cues.
The paper tackles the problem of visual relationship detection, which suffers from combinatorial explosion, by modeling object interactions as a graph and using a message-passing algorithm to propagate contextual information, achieving superior results on two benchmark datasets.
Visual relationship detection aims to detect the interactions between objects in an image; however, this task suffers from combinatorial explosion due to the variety of objects and interactions. Since the interactions associated with the same object are dependent, we explore the dependency of interactions to reduce the search space. We explicitly model objects and interactions by an interaction graph and then propose a message-passing-style algorithm to propagate the contextual information. We thus call the proposed method neural message passing (NMP). We further integrate language priors and spatial cues to rule out unrealistic interactions and capture spatial interactions. Experimental results on two benchmark datasets demonstrate the superiority of our proposed method. Our code is available at https://github.com/PhyllisH/NMP.