Revis: Sparse Latent Steering to Mitigate Object Hallucination in Large Vision-Language Models
This addresses a critical reliability issue for users of vision-language AI systems, offering an incremental improvement over existing methods.
The paper tackles object hallucination in Large Vision-Language Models by proposing REVIS, a training-free framework that re-activates suppressed visual information through orthogonal projection and sparse intervention, reducing hallucination rates by about 19% compared to state-of-the-art baselines.
Despite the advanced capabilities of Large Vision-Language Models (LVLMs), they frequently suffer from object hallucination. One reason is that visual features and pretrained textual representations often become intertwined in the deeper network layers. To address this, we propose REVIS, a training-free framework designed to explicitly re-activate this suppressed visual information. Rooted in latent space geometry, REVIS extracts the pure visual information vector via orthogonal projection and employs a calibrated strategy to perform sparse intervention only at the precise depth where suppression occurs. This surgical approach effectively restores visual information with minimal computational cost. Empirical evaluations on standard benchmarks demonstrate that REVIS reduces object hallucination rates by approximately 19% compared to state-of-the-art baselines, while preserving general reasoning capabilities.