Fault-Tolerant Collaborative Inference through the Edge-PRUNE Framework
This addresses fault tolerance in collaborative inference for applications like surveillance, though it appears incremental as it builds on existing dynamic inference methodologies.
The paper tackles the problem of device failure in collaborative inference systems by introducing the Edge-PRUNE framework, which achieves fault tolerance with experimental results showing inference time savings and execution time overhead analysis.
Collaborative inference has received significant research interest in machine learning as a vehicle for distributing computation load, reducing latency, as well as addressing privacy preservation in communications. Recent collaborative inference frameworks have adopted dynamic inference methodologies such as early-exit and run-time partitioning of neural networks. However, as machine learning frameworks scale in the number of inference inputs, e.g., in surveillance applications, fault tolerance related to device failure needs to be considered. This paper presents the Edge-PRUNE distributed computing framework, built on a formally defined model of computation, which provides a flexible infrastructure for fault tolerant collaborative inference. The experimental section of this work shows results on achievable inference time savings by collaborative inference, presents fault tolerant system topologies and analyzes their cost in terms of execution time overhead.