VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning
For practitioners of multimodal large language models, VCap provides a fine-grained reward signal that improves captioning accuracy and generalization, addressing the bottleneck of reliable factual verification in RL-based training.
VCap introduces a Witness-Adjudicator reward mechanism for visual captioning that uses hypergeometric-distribution-level precision to verify factual consistency between generated captions and visual signals, enabling weak-to-strong RL training. An 8B model trained with VCap outperforms open- and closed-source SOTA models on multiple image and video captioning benchmarks.
Visual captioning requires models to capture visual content faithfully while minimizing both omission and hallucination. As the dominant paradigm for captioning, MLLMs have achieved strong performance through scaling and high-quality data. Recently, RL has emerged as a key route to driving MLLMs toward higher precision and broader coverage, however, existing reward designs for captioning fail to provide fine-grained and reliable signals for factual verification, limiting their effectiveness. To address this, we propose VCap, a Witness-Adjudicator reward that pairs the reference caption (a witness) with the visual signal (an adjudicator). By explicitly verifying factual consistency between the reference and policy-generated captions grounded in the visual signal, VCap delivers a reward signal with hypergeometric-distribution-level precision for caption quality verification. This design enables effective learning even from imperfect references, facilitating weak-to-strong generalization in RL training. In our experiments, an 8B model trained with VCap outperforms open- and closed-source SOTA models on multiple image and video captioning benchmarks. Human evaluation further confirms its strong alignment with factual correctness. Additionally, VCap improves MLLM perceptual capability, generalizes across tasks, and surpasses best-of-N distillation, challenging prior assumptions about RLVR.