Fully Automatic Video Colorization with Self-Regularization and Diversity
This addresses the problem of colorizing videos automatically for applications in media restoration and enhancement, representing an incremental improvement over existing methods.
The paper tackles fully automatic video colorization without labeled data by using self-regularization and diversity, achieving results that outperform state-of-the-art approaches.
We present a fully automatic approach to video colorization with self-regularization and diversity. Our model contains a colorization network for video frame colorization and a refinement network for spatiotemporal color refinement. Without any labeled data, both networks can be trained with self-regularized losses defined in bilateral and temporal space. The bilateral loss enforces color consistency between neighboring pixels in a bilateral space and the temporal loss imposes constraints between corresponding pixels in two nearby frames. While video colorization is a multi-modal problem, our method uses a perceptual loss with diversity to differentiate various modes in the solution space. Perceptual experiments demonstrate that our approach outperforms state-of-the-art approaches on fully automatic video colorization. The results are shown in the supplementary video at https://youtu.be/Y15uv2jnK-4