Injecting Distributional Awareness into MLLMs via Reinforcement Learning for Deep Imbalanced Regression
This work addresses the problem of deep imbalanced regression for MLLMs, which is a known bottleneck in multimodal AI, and provides a plug-and-play solution that improves tail performance without architectural changes.
Multimodal large language models (MLLMs) suffer from poor performance on numerical regression with long-tailed target distributions due to bias toward high-density regions. The authors propose a distribution-aware reinforcement learning framework using Group Relative Policy Optimization and a Concordance Correlation Coefficient-based reward, achieving consistent improvements over SFT and existing methods, especially in medium- and few-shot regimes.
Multimodal large language models (MLLMs) struggle with numerical regression under long-tailed target distributions. Token-level supervised fine-tuning (SFT) and point-wise regression rewards bias learning toward high-density regions, leading to regression-to-the-mean behavior and poor tail performance. We identify the lack of cross-sample relational supervision as a key limitation of existing MLLM training paradigms. To address it, we propose a distribution-aware reinforcement learning framework based on Group Relative Policy Optimization, which introduces batch-level comparison-based supervision via the Concordance Correlation Coefficient-based reward to align predicted and ground-truth distributions in terms of correlation, scale, and mean. The framework is plug-and-play, requiring no architectural modification. Experiments on a unified suite of long-tailed regression benchmarks show consistent improvements over SFT and existing MLLM regression methods, with particularly strong gains in medium- and few-shot regimes.