FVQ: A Large-Scale Dataset and an LMM-based Method for Face Video Quality Assessment
This work addresses the problem of assessing face video quality for social media platforms and human visual systems, representing an incremental advancement by applying existing LMM techniques to a new domain-specific dataset.
The authors tackled the lack of large-scale datasets for face video quality assessment (FVQA) by introducing FVQ-20K, a dataset with 20,000 in-the-wild face videos and MOS annotations, and proposed FVQ-Rater, an LMM-based method that achieved superior performance on FVQ-20K and CFVQA datasets.
Face video quality assessment (FVQA) deserves to be explored in addition to general video quality assessment (VQA), as face videos are the primary content on social media platforms and human visual system (HVS) is particularly sensitive to human faces. However, FVQA is rarely explored due to the lack of large-scale FVQA datasets. To fill this gap, we present the first large-scale in-the-wild FVQA dataset, FVQ-20K, which contains 20,000 in-the-wild face videos together with corresponding mean opinion score (MOS) annotations. Along with the FVQ-20K dataset, we further propose a specialized FVQA method named FVQ-Rater to achieve human-like rating and scoring for face video, which is the first attempt to explore the potential of large multimodal models (LMMs) for the FVQA task. Concretely, we elaborately extract multi-dimensional features including spatial features, temporal features, and face-specific features (i.e., portrait features and face embeddings) to provide comprehensive visual information, and take advantage of the LoRA-based instruction tuning technique to achieve quality-specific fine-tuning, which shows superior performance on both FVQ-20K and CFVQA datasets. Extensive experiments and comprehensive analysis demonstrate the significant potential of the FVQ-20K dataset and FVQ-Rater method in promoting the development of FVQA.