Hallucination Detection for Grounded Instruction Generation
This addresses the issue of unreliable instruction generation for human navigation in simulated environments, but it is incremental as it builds on existing pre-trained models and focuses on a specific detection task.
The paper tackles the problem of hallucination in models generating navigation instructions for simulated residential environments, where models produce inconsistent references to actions or objects. The result is a model that detects these hallucinations by fine-tuning a pre-trained image-text model with contrastive loss, outperforming baselines like word probability estimates and supervised LSTM/Transformer models.
We investigate the problem of generating instructions to guide humans to navigate in simulated residential environments. A major issue with current models is hallucination: they generate references to actions or objects that are inconsistent with what a human follower would perform or encounter along the described path. We develop a model that detects these hallucinated references by adopting a model pre-trained on a large corpus of image-text pairs, and fine-tuning it with a contrastive loss that separates correct instructions from instructions containing synthesized hallucinations. Our final model outperforms several baselines, including using word probability estimated by the instruction-generation model, and supervised models based on LSTM and Transformer.