AIVD: Adaptive Edge-Cloud Collaboration for Accurate and Efficient Industrial Visual Detection
This work addresses resource-constrained edge-cloud deployment for industrial visual detection, offering incremental improvements in efficiency and accuracy.
The paper tackles the challenge of precise object localization and efficient deployment of multimodal large language models (MLLMs) in industrial visual detection by proposing the AIVD framework, which uses edge-cloud collaboration to reduce resource consumption and improve classification accuracy and semantic generation quality.
Multimodal large language models (MLLMs) demonstrate exceptional capabilities in semantic understanding and visual reasoning, yet they still face challenges in precise object localization and resource-constrained edge-cloud deployment. To address this, this paper proposes the AIVD framework, which achieves unified precise localization and high-quality semantic generation through the collaboration between lightweight edge detectors and cloud-based MLLMs. To enhance the cloud MLLM's robustness against edge cropped-box noise and scenario variations, we design an efficient fine-tuning strategy with visual-semantic collaborative augmentation, significantly improving classification accuracy and semantic consistency. Furthermore, to maintain high throughput and low latency across heterogeneous edge devices and dynamic network conditions, we propose a heterogeneous resource-aware dynamic scheduling algorithm. Experimental results demonstrate that AIVD substantially reduces resource consumption while improving MLLM classification performance and semantic generation quality. The proposed scheduling strategy also achieves higher throughput and lower latency across diverse scenarios.