Efficient Model-Based Deep Learning via Network Pruning and Fine-Tuning
This work addresses scalability issues in MBDL for inverse problems, offering incremental improvements in efficiency for applications like medical imaging or remote sensing.
The paper tackles the high computational cost of model-based deep learning (MBDL) networks for imaging inverse problems by applying structured pruning and fine-tuning, achieving speedups of 50% and 32% for deep equilibrium learning and deep unfolding methods with minimal performance loss.
Model-based deep learning (MBDL) is a powerful methodology for designing deep models to solve imaging inverse problems. MBDL networks can be seen as iterative algorithms that estimate the desired image using a physical measurement model and a learned image prior specified using a convolutional neural net (CNNs). The iterative nature of MBDL networks increases the test-time computational complexity, which limits their applicability in certain large-scale applications. Here we make two contributions to address this issue: First, we show how structured pruning can be adopted to reduce the number of parameters in MBDL networks. Second, we present three methods to fine-tune the pruned MBDL networks to mitigate potential performance loss. Each fine-tuning strategy has a unique benefit that depends on the presence of a pre-trained model and a high-quality ground truth. We show that our pruning and fine-tuning approach can accelerate image reconstruction using popular deep equilibrium learning (DEQ) and deep unfolding (DU) methods by 50% and 32%, respectively, with nearly no performance loss. This work thus offers a step forward for solving inverse problems by showing the potential of pruning to improve the scalability of MBDL. Code is available at https://github.com/wustl-cig/MBDL_Pruning .