Breaking the accuracy-resource dilemma: a lightweight adaptive video inference enhancement
This addresses resource-constrained video inference for edge devices, but it is incremental as it builds on existing methods with adaptive scaling.
The paper tackles the trade-off between accuracy and resource efficiency in video inference by developing a fuzzy controller and framework that dynamically switches between models based on real-time conditions, achieving a balance with improved performance.
Existing video inference (VI) enhancement methods typically aim to improve performance by scaling up model sizes and employing sophisticated network architectures. While these approaches demonstrated state-of-the-art performance, they often overlooked the trade-off of resource efficiency and inference effectiveness, leading to inefficient resource utilization and suboptimal inference performance. To address this problem, a fuzzy controller (FC-r) is developed based on key system parameters and inference-related metrics. Guided by the FC-r, a VI enhancement framework is proposed, where the spatiotemporal correlation of targets across adjacent video frames is leveraged. Given the real-time resource conditions of the target device, the framework can dynamically switch between models of varying scales during VI. Experimental results demonstrate that the proposed method effectively achieves a balance between resource utilization and inference performance.