FixyNN: Efficient Hardware for Mobile Computer Vision via Transfer Learning
This addresses energy efficiency for mobile computer vision applications, offering a novel hardware-software co-design that is incremental but provides substantial gains.
The paper tackles the high computational demands of CNN-based computer vision on mobile devices by proposing FixyNN, a hardware design with a fixed-weight feature extractor and a programmable CNN accelerator, achieving up to 26.6 TOPS/W energy efficiency and less than 1% accuracy loss across six datasets.
The computational demands of computer vision tasks based on state-of-the-art Convolutional Neural Network (CNN) image classification far exceed the energy budgets of mobile devices. This paper proposes FixyNN, which consists of a fixed-weight feature extractor that generates ubiquitous CNN features, and a conventional programmable CNN accelerator which processes a dataset-specific CNN. Image classification models for FixyNN are trained end-to-end via transfer learning, with the common feature extractor representing the transfered part, and the programmable part being learnt on the target dataset. Experimental results demonstrate FixyNN hardware can achieve very high energy efficiencies up to 26.6 TOPS/W ($4.81 \times$ better than iso-area programmable accelerator). Over a suite of six datasets we trained models via transfer learning with an accuracy loss of $<1\%$ resulting in up to 11.2 TOPS/W - nearly $2 \times$ more efficient than a conventional programmable CNN accelerator of the same area.