Causal Decoding for Hallucination-Resistant Multimodal Large Language Models
This addresses reliability issues in MLLMs for vision-language tasks, offering a targeted solution to a known bottleneck, though it is incremental in improving existing methods.
The paper tackled the problem of object hallucination in Multimodal Large Language Models (MLLMs), where models introduce objects not present in images, and proposed a causal decoding framework that reduces hallucination rates while maintaining output quality, achieving state-of-the-art faithfulness on captioning and QA benchmarks.
Multimodal Large Language Models (MLLMs) deliver detailed responses on vision-language tasks, yet remain susceptible to object hallucination (introducing objects not present in the image), undermining reliability in practice. Prior efforts often rely on heuristic penalties, post-hoc correction, or generic decoding tweaks, which do not directly intervene in the mechanisms that trigger object hallucination and thus yield limited gains. To address this challenge, we propose a causal decoding framework that applies targeted causal interventions during generation to curb spurious object mentions. By reshaping the decoding dynamics to attenuate spurious dependencies, our approach reduces false object tokens while maintaining descriptive quality. Across captioning and QA benchmarks, our framework substantially lowers object-hallucination rates and achieves state-of-the-art faithfulness without degrading overall output quality.