ERNet Family: Hardware-Oriented CNN Models for Computational Imaging Using Block-Based Inference
This work addresses hardware efficiency issues in computational imaging for applications like image denoising and super-resolution, though it is incremental as it builds on existing block-based methods.
The paper tackles the high DRAM bandwidth demands of CNNs in computational imaging by proposing the ERNet family, which uses block-based inference and temporary layer expansion to balance performance and hardware constraints, achieving superior image quality and throughput compared to state-of-the-art models like FFDNet and EDSR-baseline.
Convolutional neural networks (CNNs) demand huge DRAM bandwidth for computational imaging tasks, and block-based processing has recently been applied to greatly reduce the bandwidth. However, the induced additional computation for feature recomputing or the large SRAM for feature reusing will degrade the performance or even forbid the usage of state-of-the-art models. In this paper, we address these issues by considering the overheads and hardware constraints in advance when constructing CNNs. We investigate a novel model family---ERNet---which includes temporary layer expansion as another means for increasing model capacity. We analyze three ERNet variants in terms of hardware requirement and introduce a hardware-aware model optimization procedure. Evaluations on Full HD and 4K UHD applications will be given to show the effectiveness in terms of image quality, pixel throughput, and SRAM usage. The results also show that, for block-based inference, ERNet can outperform the state-of-the-art FFDNet and EDSR-baseline models for image denoising and super-resolution respectively.