Hardware-efficient Residual Networks for FPGAs
This work addresses the hardware implementation challenges of ResNets for FPGA designers, offering a method to improve efficiency.
The paper tackles the challenge of hardware inefficiency in Residual Networks (ResNets) due to their skip connections. By gradually pruning all skip connections using a teacher-student learning method, they construct a "NonResNet" which, when implemented on FPGAs, reduces BRAM utilization by 9%, LUT utilization by 3%, and increases throughput by 5%.
Residual networks (ResNets) employ skip connections in their networks -- reusing activations from previous layers -- to improve training convergence, but these skip connections create challenges for hardware implementations of ResNets. The hardware must either wait for skip connections to be processed before processing more incoming data or buffer them elsewhere. Without skip connections, ResNets would be more hardware-efficient. Thus, we present the teacher-student learning method to gradually prune away all of a ResNet's skip connections, constructing a network we call NonResNet. We show that when implemented for FPGAs, NonResNet decreases ResNet's BRAM utilization by 9% and LUT utilization by 3% and increases throughput by 5%.