Processing of incomplete images by (graph) convolutional neural networks
This addresses the issue of handling incomplete image data for computer vision applications, though it is incremental as it adapts existing graph convolutional methods to a specific data problem.
The paper tackles the problem of training neural networks on images with missing pixels by representing images as graphs to ignore missing values, using spatial graph convolutional networks (SGCN) as a generalization of CNNs. Experiments show this approach outperforms CNNs with imputation on classification and reconstruction tasks.
We investigate the problem of training neural networks from incomplete images without replacing missing values. For this purpose, we first represent an image as a graph, in which missing pixels are entirely ignored. The graph image representation is processed using a spatial graph convolutional network (SGCN) -- a type of graph convolutional networks, which is a proper generalization of classical CNNs operating on images. On one hand, our approach avoids the problem of missing data imputation while, on the other hand, there is a natural correspondence between CNNs and SGCN. Experiments confirm that our approach performs better than analogical CNNs with the imputation of missing values on typical classification and reconstruction tasks.