Hacking VMAF and VMAF NEG: vulnerability to different preprocessing methods
This exposes vulnerabilities in widely used video-quality metrics, which could mislead developers and researchers in video-processing applications.
The paper demonstrates that video preprocessing can artificially inflate VMAF and VMAF NEG quality metrics by up to 218.8% and 23.6%, respectively, while subjective comparisons show that visual quality often degrades or remains unchanged.
Video-quality measurement plays a critical role in the development of video-processing applications. In this paper, we show how video preprocessing can artificially increase the popular quality metric VMAF and its tuning-resistant version, VMAF NEG. We propose a pipeline that tunes processing-algorithm parameters to increase VMAF by up to 218.8%. A subjective comparison revealed that for most preprocessing methods, a video's visual quality drops or stays unchanged. We also show that some preprocessing methods can increase VMAF NEG scores by up to 23.6%.