An automated approach for improving the inference latency and energy efficiency of pretrained CNNs by removing irrelevant pixels with focused convolutions
This addresses energy efficiency for computer vision applications using pretrained CNNs, offering an incremental improvement by modifying existing models.
The paper tackles the problem of high energy and computation requirements in pretrained CNNs by proposing an automated method to improve inference latency and energy efficiency without retraining, achieving up to 25% latency reduction and 22% energy savings with minimal accuracy loss.
Computer vision often uses highly accurate Convolutional Neural Networks (CNNs), but these deep learning models are associated with ever-increasing energy and computation requirements. Producing more energy-efficient CNNs often requires model training which can be cost-prohibitive. We propose a novel, automated method to make a pretrained CNN more energy-efficient without re-training. Given a pretrained CNN, we insert a threshold layer that filters activations from the preceding layers to identify regions of the image that are irrelevant, i.e. can be ignored by the following layers while maintaining accuracy. Our modified focused convolution operation saves inference latency (by up to 25%) and energy costs (by up to 22%) on various popular pretrained CNNs, with little to no loss in accuracy.