Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution
This work addresses the challenge of deploying efficient super-resolution models on resource-limited mobile platforms, representing an incremental improvement through compiler-aware optimization.
The authors tackled the problem of real-time super-resolution on mobile devices by proposing a compiler-aware neural architecture search framework that optimizes for both image quality and inference speed, achieving real-time 720p SR on a Samsung Galaxy S21 with competitive PSNR and SSIM scores.
Deep learning-based super-resolution (SR) has gained tremendous popularity in recent years because of its high image quality performance and wide application scenarios. However, prior methods typically suffer from large amounts of computations and huge power consumption, causing difficulties for real-time inference, especially on resource-limited platforms such as mobile devices. To mitigate this, we propose a compiler-aware SR neural architecture search (NAS) framework that conducts depth search and per-layer width search with adaptive SR blocks. The inference speed is directly taken into the optimization along with the SR loss to derive SR models with high image quality while satisfying the real-time inference requirement. Instead of measuring the speed on mobile devices at each iteration during the search process, a speed model incorporated with compiler optimizations is leveraged to predict the inference latency of the SR block with various width configurations for faster convergence. With the proposed framework, we achieve real-time SR inference for implementing 720p resolution with competitive SR performance (in terms of PSNR and SSIM) on GPU/DSP of mobile platforms (Samsung Galaxy S21).