LCV2: An Efficient Pretraining-Free Framework for Grounded Visual Question Answering
This work addresses the challenge of efficient and adaptable grounded VQA for low-resource settings, though it is incremental as it builds on existing models.
The paper tackles the problem of grounded visual question answering by proposing LCV2, a modular framework that uses a frozen large language model to mediate between off-the-shelf VQA and visual grounding models without pre-training, achieving robust competitiveness on benchmark datasets like GQA, CLEVR, and VizWiz-VQA-Grounding.
In this paper, the LCV2 modular method is proposed for the Grounded Visual Question Answering task in the vision-language multimodal domain. This approach relies on a frozen large language model (LLM) as intermediate mediator between the off-the-shelf VQA model and the off-the-shelf visual grounding (VG) model, where the LLM transforms and conveys textual information between the two modules based on a designed prompt. LCV2 establish an integrated plug-and-play framework without the need for any pre-training process. This framework can be deployed for VQA Grounding tasks under low computational resources. The modularized model within the framework allows application with various state-of-the-art pre-trained models, exhibiting significant potential to be advance with the times. Experimental implementations were conducted under constrained computational and memory resources, evaluating the proposed method's performance on benchmark datasets including GQA, CLEVR, and VizWiz-VQA-Grounding. Comparative analyses with baseline methods demonstrate the robust competitiveness of LCV2.