Real-Time Edge Classification: Optimal Offloading under Token Bucket Constraints
This work addresses latency constraints in edge computing for real-time applications, offering a solution for devices to efficiently offload tasks while managing network resources, though it is incremental in its approach.
The paper tackles the problem of real-time edge classification by developing a framework for optimal offloading decisions under token bucket constraints to manage network delays, achieving minimized error rates on the ImageNet benchmark.
To deploy machine learning-based algorithms for real-time applications with strict latency constraints, we consider an edge-computing setting where a subset of inputs are offloaded to the edge for processing by an accurate but resource-intensive model, and the rest are processed only by a less-accurate model on the device itself. Both models have computational costs that match available compute resources, and process inputs with low-latency. But offloading incurs network delays, and to manage these delays to meet application deadlines, we use a token bucket to constrain the average rate and burst length of transmissions from the device. We introduce a Markov Decision Process-based framework to make offload decisions under these constraints, based on the local model's confidence and the token bucket state, with the goal of minimizing a specified error measure for the application. Beyond isolated decisions for individual devices, we also propose approaches to allow multiple devices connected to the same access switch to share their bursting allocation. We evaluate and analyze the policies derived using our framework on the standard ImageNet image classification benchmark.