An Efficient Recurrent Adversarial Framework for Unsupervised Real-Time Video Enhancement
This work addresses the problem of real-time video enhancement for users and applications requiring high-quality video processing without paired training data, offering an incremental improvement over existing methods.
This paper tackles the problem of unsupervised real-time video enhancement, which is challenging due to computational cost, data volume, spatio-temporal consistency, and lack of paired examples. The proposed framework, utilizing novel recurrent cells and a single discriminator, achieves superior visual quality, quantitative metrics, and inference speed, notably enhancing over 35 frames per second of FullHD video.
Video enhancement is a challenging problem, more than that of stills, mainly due to high computational cost, larger data volumes and the difficulty of achieving consistency in the spatio-temporal domain. In practice, these challenges are often coupled with the lack of example pairs, which inhibits the application of supervised learning strategies. To address these challenges, we propose an efficient adversarial video enhancement framework that learns directly from unpaired video examples. In particular, our framework introduces new recurrent cells that consist of interleaved local and global modules for implicit integration of spatial and temporal information. The proposed design allows our recurrent cells to efficiently propagate spatio-temporal information across frames and reduces the need for high complexity networks. Our setting enables learning from unpaired videos in a cyclic adversarial manner, where the proposed recurrent units are employed in all architectures. Efficient training is accomplished by introducing one single discriminator that learns the joint distribution of source and target domain simultaneously. The enhancement results demonstrate clear superiority of the proposed video enhancer over the state-of-the-art methods, in all terms of visual quality, quantitative metrics, and inference speed. Notably, our video enhancer is capable of enhancing over 35 frames per second of FullHD video (1080x1920).