FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoation
This work addresses the need for better quality metrics in VFI, which is important for video processing and compression applications, though it is incremental as it builds on an existing method.
The paper tackles the problem of perceptual quality assessment for video frame interpolation (VFI) by introducing FloLPIPS, a full-reference video quality metric that modifies LPIPS with temporal distortion weighting, achieving superior correlation with subjective ground truth on the BVI-VFI database over 12 existing assessors.
Video frame interpolation (VFI) serves as a useful tool for many video processing applications. Recently, it has also been applied in the video compression domain for enhancing both conventional video codecs and learning-based compression architectures. While there has been an increased focus on the development of enhanced frame interpolation algorithms in recent years, the perceptual quality assessment of interpolated content remains an open field of research. In this paper, we present a bespoke full reference video quality metric for VFI, FloLPIPS, that builds on the popular perceptual image quality metric, LPIPS, which captures the perceptual degradation in extracted image feature space. In order to enhance the performance of LPIPS for evaluating interpolated content, we re-designed its spatial feature aggregation step by using the temporal distortion (through comparing optical flows) to weight the feature difference maps. Evaluated on the BVI-VFI database, which contains 180 test sequences with various frame interpolation artefacts, FloLPIPS shows superior correlation performance (with statistical significance) with subjective ground truth over 12 popular quality assessors. To facilitate further research in VFI quality assessment, our code is publicly available at https://danier97.github.io/FloLPIPS.