On Data Synthesis and Post-training for Visual Abstract Reasoning
This addresses the problem of poor AVR performance in VLMs, representing a breakthrough in a challenging domain, though it is an early effort in this specific area.
The paper tackles abstract visual reasoning (AVR) for large vision-language models (VLMs) by enabling a LLaVA-NeXT 7B model to outperform larger models like Qwen-2-VL-72B and GPT-4o on AVR benchmarks, achieving significant performance gains.
This paper is a pioneering work attempting to address abstract visual reasoning (AVR) problems for large vision-language models (VLMs). We make a common LLaVA-NeXT 7B model capable of perceiving and reasoning about specific AVR problems, surpassing both open-sourced (e.g., Qwen-2-VL-72B) and closed-sourced powerful VLMs (e.g., GPT-4o) with significant margin. This is a great breakthrough since almost all previous VLMs fail or show nearly random performance on representative AVR benchmarks. Our key success is our innovative data synthesis and post-training process, aiming to fully relieve the task difficulty and elicit the model to learn, step by step. Our 7B model is also shown to be behave well on AVR without sacrificing common multimodal comprehension abilities. We hope our paper could serve as an early effort in this area and would inspire further research in abstract visual reasoning.