Bringing Old Films Back to Life
This work addresses the restoration of old films, which is a domain-specific problem for archivists and media professionals, though it appears incremental as it builds on existing video restoration techniques.
The authors tackled the problem of restoring heavily degraded old films by developing a recurrent transformer network (RTN) that leverages information from adjacent frames to improve restoration quality and temporal coherency, achieving significant superiority over existing solutions in experiments on synthetic and real-world datasets.
We present a learning-based framework, recurrent transformer network (RTN), to restore heavily degraded old films. Instead of performing frame-wise restoration, our method is based on the hidden knowledge learned from adjacent frames that contain abundant information about the occlusion, which is beneficial to restore challenging artifacts of each frame while ensuring temporal coherency. Moreover, contrasting the representation of the current frame and the hidden knowledge makes it possible to infer the scratch position in an unsupervised manner, and such defect localization generalizes well to real-world degradations. To better resolve mixed degradation and compensate for the flow estimation error during frame alignment, we propose to leverage more expressive transformer blocks for spatial restoration. Experiments on both synthetic dataset and real-world old films demonstrate the significant superiority of the proposed RTN over existing solutions. In addition, the same framework can effectively propagate the color from keyframes to the whole video, ultimately yielding compelling restored films. The implementation and model will be released at https://github.com/raywzy/Bringing-Old-Films-Back-to-Life.