HyperCUT: Video Sequence from a Single Blurry Image using Unsupervised Ordering
This work solves the order-ambiguity issue in image-to-video deblurring, enabling more reliable training for applications in video restoration, but it is incremental as it builds on prior methods by adding explicit ordering.
The paper tackles the problem of training image-to-video deblurring models by addressing frame ordering ambiguity, proposing a self-supervised ordering scheme that maps sequences to a latent space to define explicit orders, and introduces a real-image dataset covering domains like face, hand, and street, with extensive experiments confirming its effectiveness.
We consider the challenging task of training models for image-to-video deblurring, which aims to recover a sequence of sharp images corresponding to a given blurry image input. A critical issue disturbing the training of an image-to-video model is the ambiguity of the frame ordering since both the forward and backward sequences are plausible solutions. This paper proposes an effective self-supervised ordering scheme that allows training high-quality image-to-video deblurring models. Unlike previous methods that rely on order-invariant losses, we assign an explicit order for each video sequence, thus avoiding the order-ambiguity issue. Specifically, we map each video sequence to a vector in a latent high-dimensional space so that there exists a hyperplane such that for every video sequence, the vectors extracted from it and its reversed sequence are on different sides of the hyperplane. The side of the vectors will be used to define the order of the corresponding sequence. Last but not least, we propose a real-image dataset for the image-to-video deblurring problem that covers a variety of popular domains, including face, hand, and street. Extensive experimental results confirm the effectiveness of our method. Code and data are available at https://github.com/VinAIResearch/HyperCUT.git