Revisiting DNN Training for Intermittently-Powered Energy-Harvesting Micro-Computers
This addresses the problem of deploying DNNs in energy-constrained environments like wireless sensor networks, representing an incremental advance with specific optimizations for intermittent power.
This study tackled the challenge of training deep neural networks on intermittently-powered energy-harvesting micro-computers by introducing a dynamic dropout technique that adapts to device architecture and energy variability, resulting in 6 to 22 percent accuracy improvements with less than 5 percent additional compute compared to state-of-the-art methods.
The deployment of Deep Neural Networks in energy-constrained environments, such as Energy Harvesting Wireless Sensor Networks, presents unique challenges, primarily due to the intermittent nature of power availability. To address these challenges, this study introduces and evaluates a novel training methodology tailored for DNNs operating within such contexts. In particular, we propose a dynamic dropout technique that adapts to both the architecture of the device and the variability in energy availability inherent in energy harvesting scenarios. Our proposed approach leverages a device model that incorporates specific parameters of the network architecture and the energy harvesting profile to optimize dropout rates dynamically during the training phase. By modulating the network's training process based on predicted energy availability, our method not only conserves energy but also ensures sustained learning and inference capabilities under power constraints. Our preliminary results demonstrate that this strategy provides 6 to 22 percent accuracy improvements compared to the state of the art with less than 5 percent additional compute. This paper details the development of the device model, describes the integration of energy profiles with intermittency aware dropout and quantization algorithms, and presents a comprehensive evaluation of the proposed approach using real-world energy harvesting data.