Structured Prompting and Multi-Agent Knowledge Distillation for Traffic Video Interpretation and Risk Inference
This work addresses the need for efficient and real-time traffic risk monitoring in Intelligent Transportation Systems and autonomous driving, though it is incremental as it builds on existing vision-language models and knowledge distillation techniques.
The paper tackles the problem of scalable and generalizable traffic video interpretation and risk inference by introducing a structured prompting and knowledge distillation framework, resulting in a compact 3B-scale model (VISTA) that achieves strong performance on captioning metrics like BLEU-4, METEOR, ROUGE-L, and CIDEr compared to teacher models.
Comprehensive highway scene understanding and robust traffic risk inference are vital for advancing Intelligent Transportation Systems (ITS) and autonomous driving. Traditional approaches often struggle with scalability and generalization, particularly under the complex and dynamic conditions of real-world environments. To address these challenges, we introduce a novel structured prompting and knowledge distillation framework that enables automatic generation of high-quality traffic scene annotations and contextual risk assessments. Our framework orchestrates two large Vision-Language Models (VLMs): GPT-4o and o3-mini, using a structured Chain-of-Thought (CoT) strategy to produce rich, multi-perspective outputs. These outputs serve as knowledge-enriched pseudo-annotations for supervised fine-tuning of a much smaller student VLM. The resulting compact 3B-scale model, named VISTA (Vision for Intelligent Scene and Traffic Analysis), is capable of understanding low-resolution traffic videos and generating semantically faithful, risk-aware captions. Despite its significantly reduced parameter count, VISTA achieves strong performance across established captioning metrics (BLEU-4, METEOR, ROUGE-L, and CIDEr) when benchmarked against its teacher models. This demonstrates that effective knowledge distillation and structured multi-agent supervision can empower lightweight VLMs to capture complex reasoning capabilities. The compact architecture of VISTA facilitates efficient deployment on edge devices, enabling real-time risk monitoring without requiring extensive infrastructure upgrades.