Is It a Plausible Colour? UCapsNet for Image Colourisation
This work aims to improve the plausibility and vibrancy of colours in automated image colourisation, which is a problem for computer vision systems seeking to replicate human visual understanding.
This paper addresses the image colourisation problem by proposing UCapsNet, a novel self-supervised architecture combining Capsule Networks for semantic feature extraction and Convolutional Neural Networks for spatial details. The model generates more vibrant and plausible colours, outperforming existing supervised pre-trained models on the ImageNet benchmark.
Human beings can imagine the colours of a grayscale image with no particular effort thanks to their ability of semantic feature extraction. Can an autonomous system achieve that? Can it hallucinate plausible and vibrant colours? This is the colourisation problem. Different from existing works relying on convolutional neural network models pre-trained with supervision, we cast such colourisation problem as a self-supervised learning task. We tackle the problem with the introduction of a novel architecture based on Capsules trained following the adversarial learning paradigm. Capsule networks are able to extract a semantic representation of the entities in the image but loose details about their spatial information, which is important for colourising a grayscale image. Thus our UCapsNet structure comes with an encoding phase that extracts entities through capsules and spatial details through convolutional neural networks. A decoding phase merges the entity features with the spatial features to hallucinate a plausible colour version of the input datum. Results on the ImageNet benchmark show that our approach is able to generate more vibrant and plausible colours than exiting solutions and achieves superior performance than models pre-trained with supervision.