Image-Graph-Image Translation via Auto-Encoding
This work addresses the problem of requiring meticulous annotations for image-to-graph translation, which is important for scene understanding, by proposing a self-supervised approach.
This paper introduces the first self-supervised convolutional neural network for image-to-graph translation, eliminating the need for external supervision. It successfully encodes simple line drawings into graphs, achieving results comparable to a fully-supervised baseline in terms of F1 score on triplet matching.
This work presents the first convolutional neural network that learns an image-to-graph translation task without needing external supervision. Obtaining graph representations of image content, where objects are represented as nodes and their relationships as edges, is an important task in scene understanding. Current approaches follow a fully-supervised approach thereby requiring meticulous annotations. To overcome this, we are the first to present a self-supervised approach based on a fully-differentiable auto-encoder in which the bottleneck encodes the graph's nodes and edges. This self-supervised approach can currently encode simple line drawings into graphs and obtains comparable results to a fully-supervised baseline in terms of F1 score on triplet matching. Besides these promising results, we provide several directions for future research on how our approach can be extended to cover more complex imagery.