Chain-of-Anomaly Thoughts with Large Vision-Language Models
This addresses the problem of crime detection in video surveillance for security applications, and it is incremental as it builds on existing Chain-of-Thought reasoning strategies.
The paper tackled the problem of large vision-language models failing to detect crimes in automated video surveillance due to bias towards normality, and the result was a proposed multi-agent reasoning framework called Chain-of-Anomaly-Thoughts that improved anomaly detection F1-score by 11.8 percentage points on low-resolution footage and anomaly classification by 3.78 percentage points in high-resolution videos.
Automated video surveillance with Large Vision-Language Models is limited by their inherent bias towards normality, often failing to detect crimes. While Chain-of-Thought reasoning strategies show significant potential for improving performance in language tasks, the lack of inductive anomaly biases in their reasoning further steers the models towards normal interpretations. To address this, we propose Chain-of-Anomaly-Thoughts (CoAT), a multi-agent reasoning framework that introduces inductive criminal bias in the reasoning process through a final, anomaly-focused classification layer. Our method significantly improves Anomaly Detection, boosting F1-score by 11.8 p.p. on challenging low-resolution footage and Anomaly Classification by 3.78 p.p. in high-resolution videos.