Representative Graph Neural Network
This addresses computational inefficiency in graph neural networks for computer vision tasks, offering a flexible solution for integrating into various architectures, though it is incremental in optimizing existing methods.
The paper tackles the prohibitive computational complexity of non-local operations in modeling long-range dependencies by introducing a Representative Graph (RepGraph) layer that dynamically samples a few representative features, reducing redundancy. It achieves competitive or favorable performance against state-of-the-art methods on semantic segmentation benchmarks like ADE20K, Cityscapes, and PASCAL-Context, and improves object detection on COCO compared to non-local operations.
Non-local operation is widely explored to model the long-range dependencies. However, the redundant computation in this operation leads to a prohibitive complexity. In this paper, we present a Representative Graph (RepGraph) layer to dynamically sample a few representative features, which dramatically reduces redundancy. Instead of propagating the messages from all positions, our RepGraph layer computes the response of one node merely with a few representative nodes. The locations of representative nodes come from a learned spatial offset matrix. The RepGraph layer is flexible to integrate into many visual architectures and combine with other operations. With the application of semantic segmentation, without any bells and whistles, our RepGraph network can compete or perform favourably against the state-of-the-art methods on three challenging benchmarks: ADE20K, Cityscapes, and PASCAL-Context datasets. In the task of object detection, our RepGraph layer can also improve the performance on the COCO dataset compared to the non-local operation. Code is available at https://git.io/RepGraph.