Uncertainty-Aware Mean Opinion Score Prediction
This work addresses the practical application of MOS prediction systems in real and open-world environments, though it appears incremental as it builds on existing uncertainty modeling techniques.
The paper tackled the problem of unstable performance in Mean Opinion Score (MOS) prediction models across diverse samples by proposing an uncertainty-aware system that models aleatory and epistemic uncertainty using heteroscedastic regression and Monte Carlo dropout, resulting in a system that captures uncertainty well and enables selective prediction and out-of-domain detection.
Mean Opinion Score (MOS) prediction has made significant progress in specific domains. However, the unstable performance of MOS prediction models across diverse samples presents ongoing challenges in the practical application of these systems. In this paper, we point out that the absence of uncertainty modeling is a significant limitation hindering MOS prediction systems from applying to the real and open world. We analyze the sources of uncertainty in the MOS prediction task and propose to establish an uncertainty-aware MOS prediction system that models aleatory uncertainty and epistemic uncertainty by heteroscedastic regression and Monte Carlo dropout separately. The experimental results show that the system captures uncertainty well and is capable of performing selective prediction and out-of-domain detection. Such capabilities significantly enhance the practical utility of MOS systems in diverse real and open-world environments.