Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models
This work addresses the critical need for reliable ATR in military and surveillance applications, where safety and accuracy are challenged by novel terrains and objects, representing an incremental improvement by integrating existing methods.
The paper tackles the problem of automatic target recognition (ATR) in unknown environments and for novel object classes by proposing a pipeline that combines open-world detectors with large vision-language models (LVLMs) for zero-shot scene understanding, achieving robust performance in recognizing military vehicles under varying conditions.
Automatic target recognition (ATR) plays a critical role in tasks such as navigation and surveillance, where safety and accuracy are paramount. In extreme use cases, such as military applications, these factors are often challenged due to the presence of unknown terrains, environmental conditions, and novel object categories. Current object detectors, including open-world detectors, lack the ability to confidently recognize novel objects or operate in unknown environments, as they have not been exposed to these new conditions. However, Large Vision-Language Models (LVLMs) exhibit emergent properties that enable them to recognize objects in varying conditions in a zero-shot manner. Despite this, LVLMs struggle to localize objects effectively within a scene. To address these limitations, we propose a novel pipeline that combines the detection capabilities of open-world detectors with the recognition confidence of LVLMs, creating a robust system for zero-shot ATR of novel classes and unknown domains. In this study, we compare the performance of various LVLMs for recognizing military vehicles, which are often underrepresented in training datasets. Additionally, we examine the impact of factors such as distance range, modality, and prompting methods on the recognition performance, providing insights into the development of more reliable ATR systems for novel conditions and classes.