Hybrid, Unified and Iterative: A Novel Framework for Text-based Person Anomaly Retrieval
This work addresses the problem of fine-grained retrieval for security or surveillance applications, but it appears incremental as it builds on existing vision-language models and ensemble methods.
The paper tackles text-based person anomaly retrieval by proposing a hybrid framework integrating local-global features and iterative ensemble strategies, achieving state-of-the-art performance with improvements of 9.70% in R@1, 1.77% in R@5, and 1.01% in R@10 on the PAB dataset.
Text-based person anomaly retrieval has emerged as a challenging task, with most existing approaches relying on complex deep-learning techniques. This raises a research question: How can the model be optimized to achieve greater fine-grained features? To address this, we propose a Local-Global Hybrid Perspective (LHP) module integrated with a Vision-Language Model (VLM), designed to explore the effectiveness of incorporating both fine-grained features alongside coarse-grained features. Additionally, we investigate a Unified Image-Text (UIT) model that combines multiple objective loss functions, including Image-Text Contrastive (ITC), Image-Text Matching (ITM), Masked Language Modeling (MLM), and Masked Image Modeling (MIM) loss. Beyond this, we propose a novel iterative ensemble strategy, by combining iteratively instead of using model results simultaneously like other ensemble methods. To take advantage of the superior performance of the LHP model, we introduce a novel feature selection algorithm based on its guidance, which helps improve the model's performance. Extensive experiments demonstrate the effectiveness of our method in achieving state-of-the-art (SOTA) performance on PAB dataset, compared with previous work, with a 9.70\% improvement in R@1, 1.77\% improvement in R@5, and 1.01\% improvement in R@10.