Exploring Energy-Accuracy Tradeoffs in AI Hardware
This work addresses the critical problem of energy efficiency for AI on edge devices, which is important for mobile, autonomous, and sensor applications.
This paper explores the scenario where AI systems on edge devices must operate at reduced accuracy to meet energy constraints. It proposes a cost function for AI systems and demonstrates that minimizing this cost for binary decision problems with CNNs involves using fewer resources. The study shows that energy costs can be significantly reduced by leveraging high-confidence predictions from earlier network layers.
Artificial intelligence (AI) is playing an increasingly significant role in our everyday lives. This trend is expected to continue, especially with recent pushes to move more AI to the edge. However, one of the biggest challenges associated with AI on edge devices (mobile phones, unmanned vehicles, sensors, etc.) is their associated size, weight, and power constraints. In this work, we consider the scenario where an AI system may need to operate at less-than-maximum accuracy in order to meet application-dependent energy requirements. We propose a simple function that divides the cost of using an AI system into the cost of the decision making process and the cost of decision execution. For simple binary decision problems with convolutional neural networks, it is shown that minimizing the cost corresponds to using fewer than the maximum number of resources (e.g. convolutional neural network layers and filters). Finally, it is shown that the cost associated with energy can be significantly reduced by leveraging high-confidence predictions made in lower-level layers of the network.