José Gallego

Reza Namvar, José Gallego, Jose A. Ayala-Romero et al.

Network operation relies on heuristics to solve many tasks rapidly and efficiently across the protocol stack. These heuristics are the result of thorough human-driven design rooted in expert knowledge of the target system and problem. Recently, approaches powered by Artificial Intelligence have shown promising results in devising solutions that outperform long-established heuristics in classical problems. We explore the possibility of applying such Automated Heuristic Design (AHD) frameworks to network environments by (i) discussing the general integration of AHD with network operation and the associated challenges, as well as (ii) proposing a practical implementation of AHD for a specific networking task, i.e., 5G decoding. Initial results show how modern AHD tools can devise heuristics for Low-Density Parity Check decoding on par with state-of-the-art solutions implemented in production systems.

Adarsh Prasad Behera, Paulius Daubaris, Iñaki Bravo et al.

On-device inference holds great potential for increased energy efficiency, responsiveness, and privacy in edge ML systems. However, due to less capable ML models that can be embedded in resource-limited devices, use cases are limited to simple inference tasks such as visual keyword spotting, gesture recognition, and predictive analytics. In this context, the Hierarchical Inference (HI) system has emerged as a promising solution that augments the capabilities of the local ML by offloading selected samples to an edge server or cloud for remote ML inference. Existing works demonstrate through simulation that HI improves accuracy. However, they do not account for the latency and energy consumption on the device, nor do they consider three key heterogeneous dimensions that characterize ML systems: hardware, network connectivity, and models. In contrast, this paper systematically compares the performance of HI with on-device inference based on measurements of accuracy, latency, and energy for running embedded ML models on five devices with different capabilities and three image classification datasets. For a given accuracy requirement, the HI systems we designed achieved up to 73% lower latency and up to 77% lower device energy consumption than an on-device inference system. The key to building an efficient HI system is the availability of small-size, reasonably accurate on-device models whose outputs can be effectively differentiated for samples that require remote inference. Despite the performance gains, HI requires on-device inference for all samples, which adds a fixed overhead to its latency and energy consumption. Therefore, we design a hybrid system, Early Exit with HI (EE-HI), and demonstrate that compared to HI, EE-HI reduces the latency by up to 59.7% and lowers the device's energy consumption by up to 60.4%.