ESAI: Efficient Split Artificial Intelligence via Early Exiting Using Neural Architecture Search
This addresses the problem of high communication costs and offline usability for IoT and mobile devices, offering an incremental improvement over existing cloud-based methods.
The paper tackles the computational challenge of deploying deep neural networks on mobile and IoT devices by proposing a framework that uses early exiting and neural architecture search to decide when to process data locally or send it to the cloud, reducing server transmissions by 60% while achieving 92% overall accuracy.
Recently, deep neural networks have been outperforming conventional machine learning algorithms in many computer vision-related tasks. However, it is not computationally acceptable to implement these models on mobile and IoT devices and the majority of devices are harnessing the cloud computing methodology in which outstanding deep learning models are responsible for analyzing the data on the server. This can bring the communication cost for the devices and make the whole system useless in those times where the communication is not available. In this paper, a new framework for deploying on IoT devices has been proposed which can take advantage of both the cloud and the on-device models by extracting the meta-information from each sample's classification result and evaluating the classification's performance for the necessity of sending the sample to the server. Experimental results show that only 40 percent of the test data should be sent to the server using this technique and the overall accuracy of the framework is 92 percent which improves the accuracy of both client and server models.