Neural Belief Propagation for Scene Graph Generation
This work addresses scene graph generation for computer vision applications, offering an incremental improvement over existing methods.
The paper tackles the problem of inconsistent interpretations in scene graph generation by proposing a neural belief propagation method that incorporates higher-order interactions, achieving state-of-the-art performance on popular benchmarks.
Scene graph generation aims to interpret an input image by explicitly modelling the potential objects and their relationships, which is predominantly solved by the message passing neural network models in previous methods. Currently, such approximation models generally assume the output variables are totally independent and thus ignore the informative structural higher-order interactions. This could lead to the inconsistent interpretations for an input image. In this paper, we propose a novel neural belief propagation method to generate the resulting scene graph. It employs a structural Bethe approximation rather than the mean field approximation to infer the associated marginals. To find a better bias-variance trade-off, the proposed model not only incorporates pairwise interactions but also higher order interactions into the associated scoring function. It achieves the state-of-the-art performance on various popular scene graph generation benchmarks.