Revisiting MLLM Based Image Quality Assessment: Errors and Remedy
This addresses a key bottleneck in using MLLMs for IQA, an incremental improvement with practical implications for image quality evaluation.
The paper tackled the mismatch between discrete token outputs of multi-modal large language models (MLLMs) and continuous quality scores in image quality assessment (IQA), proposing Q-Scorer, which achieved state-of-the-art performance across multiple IQA benchmarks.
The rapid progress of multi-modal large language models (MLLMs) has boosted the task of image quality assessment (IQA). However, a key challenge arises from the inherent mismatch between the discrete token outputs of MLLMs and the continuous nature of quality scores required by IQA tasks. This discrepancy significantly hinders the performance of MLLM-based IQA methods. Previous approaches that convert discrete token predictions into continuous scores often suffer from conversion errors. Moreover, the semantic confusion introduced by level tokens (e.g., ``good'') further constrains the performance of MLLMs on IQA tasks and degrades their original capabilities for related tasks. To tackle these problems, we provide a theoretical analysis of the errors inherent in previous approaches and, motivated by this analysis, propose a simple yet effective framework, Q-Scorer. This framework incorporates a lightweight regression module and IQA-specific score tokens into the MLLM pipeline. Extensive experiments demonstrate that Q-Scorer achieves state-of-the-art performance across multiple IQA benchmarks, generalizes well to mixed datasets, and further improves when combined with other methods.