Rank-smoothed Pairwise Learning In Perceptual Quality Assessment
This work addresses the problem of improving the accuracy of deep image quality assessment models for researchers and practitioners who rely on pairwise human preference data.
The authors propose a rank-smoothed loss function for training deep image quality assessment models using pairwise human preference data. This method regularizes empirical pairwise probabilities with aggregated rankwise probabilities, leading to more reliable training and improved accuracy in predicting human preferences.
Conducting pairwise comparisons is a widely used approach in curating human perceptual preference data. Typically raters are instructed to make their choices according to a specific set of rules that address certain dimensions of image quality and aesthetics. The outcome of this process is a dataset of sampled image pairs with their associated empirical preference probabilities. Training a model on these pairwise preferences is a common deep learning approach. However, optimizing by gradient descent through mini-batch learning means that the "global" ranking of the images is not explicitly taken into account. In other words, each step of the gradient descent relies only on a limited number of pairwise comparisons. In this work, we demonstrate that regularizing the pairwise empirical probabilities with aggregated rankwise probabilities leads to a more reliable training loss. We show that training a deep image quality assessment model with our rank-smoothed loss consistently improves the accuracy of predicting human preferences.