LVLM-COUNT: Enhancing the Counting Ability of Large Vision-Language Models
This addresses a specific weakness in LVLMs for visual counting tasks, but it is incremental as it builds on existing methods with a novel adaptation.
The paper tackled the problem of large vision-language models struggling with counting tasks, especially for large numbers of objects, by proposing a divide-and-conquer method that prevents repetitive counting, resulting in enhanced performance across various datasets.
Counting is a fundamental operation for various real-world visual tasks, requiring both object recognition and robust counting capabilities. Despite their advanced visual perception, large vision-language models (LVLMs) are known to struggle with counting tasks. In this work, we evaluate the performance of several recent LVLMs on visual counting tasks across multiple counting and vision datasets. We observe that while their performance may be less prone to error for small numbers of objects, they exhibit significant weaknesses as the number of objects increases. To alleviate this issue, we propose a simple yet effective baseline method that enhances LVLMs' counting ability for large numbers of objects using a divide-and-conquer approach. Our method decomposes counting problems into sub-tasks. Moreover, it incorporates a mechanism to prevent objects from being split during division, which could otherwise lead to repetitive counting -- a common issue in a naive divide-and-conquer implementation. We demonstrate the effectiveness of this approach across various datasets and benchmarks, establishing it as a valuable reference for evaluating future solutions.