Understanding and Evaluating Hallucinations in 3D Visual Language Models
This work addresses a critical issue for researchers and developers in embodied AI and 3D scene understanding, as hallucinations can undermine model reliability, but it is incremental as it focuses on analysis and evaluation rather than a novel solution.
The paper tackles the problem of hallucinations in 3D visual language models, where models generate non-existent objects or incorrect relationships, and it identifies three main causes—uneven object frequency, strong object correlations, and limited attribute diversity—while proposing new evaluation metrics to assess model reliance on visual information.
Recently, 3D-LLMs, which combine point-cloud encoders with large models, have been proposed to tackle complex tasks in embodied intelligence and scene understanding. In addition to showing promising results on 3D tasks, we found that they are significantly affected by hallucinations. For instance, they may generate objects that do not exist in the scene or produce incorrect relationships between objects. To investigate this issue, this work presents the first systematic study of hallucinations in 3D-LLMs. We begin by quickly evaluating hallucinations in several representative 3D-LLMs and reveal that they are all significantly affected by hallucinations. We then define hallucinations in 3D scenes and, through a detailed analysis of datasets, uncover the underlying causes of these hallucinations. We find three main causes: (1) Uneven frequency distribution of objects in the dataset. (2) Strong correlations between objects. (3) Limited diversity in object attributes. Additionally, we propose new evaluation metrics for hallucinations, including Random Point Cloud Pair and Opposite Question Evaluations, to assess whether the model generates responses based on visual information and aligns it with the text's meaning.