Relevant-features based Auxiliary Cells for Energy Efficient Detection of Natural Errors
This addresses the issue of error detection in neural networks for applications requiring energy efficiency, though it appears incremental as it builds on existing ensemble and early termination methods.
The paper tackles the problem of deep neural networks lacking the ability to detect their own prediction errors, proposing an ensemble of classifiers at hidden layers for energy-efficient detection of natural errors, with results demonstrated on datasets like CIFAR-10, CIFAR-100, and Tiny-ImageNet.
Deep neural networks have demonstrated state-of-the-art performance on many classification tasks. However, they have no inherent capability to recognize when their predictions are wrong. There have been several efforts in the recent past to detect natural errors but the suggested mechanisms pose additional energy requirements. To address this issue, we propose an ensemble of classifiers at hidden layers to enable energy efficient detection of natural errors. In particular, we append Relevant-features based Auxiliary Cells (RACs) which are class specific binary linear classifiers trained on relevant features. The consensus of RACs is used to detect natural errors. Based on combined confidence of RACs, classification can be terminated early, thereby resulting in energy efficient detection. We demonstrate the effectiveness of our technique on various image classification datasets such as CIFAR-10, CIFAR-100 and Tiny-ImageNet.