Hijacking Context in Large Multi-modal Models
This addresses a limitation in LMMs for applications requiring reliable multi-modal understanding, but it is incremental as it builds on existing models and methods.
The paper tackles the problem of Large Multi-modal Models (LMMs) being misled by incoherent images or text in their input context, causing biased outputs, and proposes a pre-filtering method using GPT-4V to remove irrelevant contexts, with further investigation into replacing hijacked contexts to improve coherence.
Recently, Large Multi-modal Models (LMMs) have demonstrated their ability to understand the visual contents of images given the instructions regarding the images. Built upon the Large Language Models (LLMs), LMMs also inherit their abilities and characteristics such as in-context learning where a coherent sequence of images and texts are given as the input prompt. However, we identify a new limitation of off-the-shelf LMMs where a small fraction of incoherent images or text descriptions mislead LMMs to only generate biased output about the hijacked context, not the originally intended context. To address this, we propose a pre-filtering method that removes irrelevant contexts via GPT-4V, based on its robustness towards distribution shift within the contexts. We further investigate whether replacing the hijacked visual and textual contexts with the correlated ones via GPT-4V and text-to-image models can help yield coherent responses.