Deep Exemplar-based Video Colorization
This work addresses the challenge of producing realistic and stable colorized videos for applications in media restoration and enhancement, representing an incremental improvement over existing methods.
The paper tackled the problem of achieving temporally consistent video colorization guided by a reference image, and introduced an end-to-end recurrent network that unifies semantic correspondence and color propagation, resulting in superior performance to state-of-the-art methods both quantitatively and qualitatively.
This paper presents the first end-to-end network for exemplar-based video colorization. The main challenge is to achieve temporal consistency while remaining faithful to the reference style. To address this issue, we introduce a recurrent framework that unifies the semantic correspondence and color propagation steps. Both steps allow a provided reference image to guide the colorization of every frame, thus reducing accumulated propagation errors. Video frames are colorized in sequence based on the colorization history, and its coherency is further enforced by the temporal consistency loss. All of these components, learned end-to-end, help produce realistic videos with good temporal stability. Experiments show our result is superior to the state-of-the-art methods both quantitatively and qualitatively.