FlowChroma -- A Deep Recurrent Neural Network for Video Colorization
This addresses color inconsistency in automated video colorization, which is an incremental improvement over existing image-based methods.
The paper tackled the problem of flickering colors in video colorization by developing a deep recurrent neural network framework, resulting in improved temporal and contextual coherence across frames as demonstrated qualitatively.
We develop an automated video colorization framework that minimizes the flickering of colors across frames. If we apply image colorization techniques to successive frames of a video, they treat each frame as a separate colorization task. Thus, they do not necessarily maintain the colors of a scene consistently across subsequent frames. The proposed solution includes a novel deep recurrent encoder-decoder architecture which is capable of maintaining temporal and contextual coherence between consecutive frames of a video. We use a high-level semantic feature extractor to automatically identify the context of a scenario including objects, with a custom fusion layer that combines the spatial and temporal features of a frame sequence. We demonstrate experimental results, qualitatively showing that recurrent neural networks can be successfully used to improve color consistency in video colorization.