SLIDE: Simultaneous Model Downloading and Inference at the Wireless Network Edge
This addresses latency issues for mobile users needing real-time on-device inference, but it is incremental as it builds on existing model downloading schemes.
The paper tackles the problem of excessive end-to-end latency for downloading and inference of large AI models at the wireless network edge by proposing the SLIDE framework, which allows simultaneous downloading and inference, and simulation results show it significantly improves task throughput under constraints.
To support on-device inference, the next-generation mobile networks are expected to support real-time model downloading services to mobile users. However, powerful AI models typically have large model sizes, resulting in excessive end-to-end (E2E) downloading-and-inference (DAI) latency. To address this issue, we propose a simultaneous model downloading and inference (SLIDE) framework, which allows users to perform inference with downloaded layers while simultaneously receiving the remaining layers of the model. To this end, we formulate a task throughput maximization problem by jointly optimizing model provisioning, spectrum bandwidth allocation, and computing resource allocation for multi-user downlink systems. Unlike traditional DAI frameworks, SLIDE introduces recursive dependencies across layers, where inference latency depends recursively on the downloading bandwidth and computing resource allocation for each of the preceding layers. To solve this challenging problem, we design an efficient algorithm that acquires the optimal solution with polynomial-time complexity. Simulation results demonstrate that the proposed SLIDE framework significantly improves task throughput under latency and communication resource constraints compared with the conventional model downloading schemes.