OmniQuality-R: Advancing Reward Models Through All-Encompassing Quality Assessment
This work addresses the need for multi-task visual evaluation in AI, but it appears incremental as it builds on existing reward modeling and optimization techniques.
The authors tackled the problem of visual evaluation being limited to single tasks by proposing OmniQuality-R, a unified reward modeling framework that transforms multi-task quality reasoning into continuous reward signals, and they evaluated it on three IQA tasks including aesthetic quality assessment, technical quality evaluation, and text-image alignment.
Current visual evaluation approaches are typically constrained to a single task. To address this, we propose OmniQuality-R, a unified reward modeling framework that transforms multi-task quality reasoning into continuous and interpretable reward signals for policy optimization. Inspired by subjective experiments, where participants are given task-specific instructions outlining distinct assessment principles prior to evaluation, we propose OmniQuality-R, a structured reward modeling framework that transforms multi-dimensional reasoning into continuous and interpretable reward signals. To enable this, we construct a reasoning-enhanced reward modeling dataset by sampling informative plan-reason trajectories via rejection sampling, forming a reliable chain-of-thought (CoT) dataset for supervised fine-tuning (SFT). Building on this, we apply Group Relative Policy Optimization (GRPO) for post-training, using a Gaussian-based reward to support continuous score prediction. To further stabilize the training and improve downstream generalization, we incorporate standard deviation (STD) filtering and entropy gating mechanisms during reinforcement learning. These techniques suppress unstable updates and reduce variance in policy optimization. We evaluate OmniQuality-R on three key IQA tasks: aesthetic quality assessment, technical quality evaluation, and text-image alignment.