Shoggoth: Towards Efficient Edge-Cloud Collaborative Real-Time Video Inference via Adaptive Online Learning
This addresses efficient video inference for edge devices with constrained resources, though it is incremental as it builds on existing collaborative and distillation methods.
The paper tackles the problem of real-time video inference under changing scenes by proposing Shoggoth, an edge-cloud collaborative architecture that uses online knowledge distillation and adaptive training, resulting in 15%-20% accuracy improvement over edge-only strategies and reduced network costs compared to cloud-only strategies.
This paper proposes Shoggoth, an efficient edge-cloud collaborative architecture, for boosting inference performance on real-time video of changing scenes. Shoggoth uses online knowledge distillation to improve the accuracy of models suffering from data drift and offloads the labeling process to the cloud, alleviating constrained resources of edge devices. At the edge, we design adaptive training using small batches to adapt models under limited computing power, and adaptive sampling of training frames for robustness and reducing bandwidth. The evaluations on the realistic dataset show 15%-20% model accuracy improvement compared to the edge-only strategy and fewer network costs than the cloud-only strategy.