Automatic Video Colorization using 3D Conditional Generative Adversarial Networks
This method addresses the problem of restoring color to old films for preservation or enhancement purposes, representing an incremental improvement by incorporating temporal consistency into video colorization.
The paper tackles automatic colorization of grayscale videos by using a 3D Conditional Generative Adversarial Network to process sequences of frames, achieving successful trials on old black-and-white films with validation through a new metric for colorization consistency.
In this work, we present a method for automatic colorization of grayscale videos. The core of the method is a Generative Adversarial Network that is trained and tested on sequences of frames in a sliding window manner. Network convolutional and deconvolutional layers are three-dimensional, with frame height, width and time as the dimensions taken into account. Multiple chrominance estimates per frame are aggregated and combined with available luminance information to recreate a colored sequence. Colorization trials are run succesfully on a dataset of old black-and-white films. The usefulness of our method is also validated with numerical results, computed with a newly proposed metric that measures colorization consistency over a frame sequence.