Learning Multiscale Convolutional Dictionaries for Image Reconstruction
This work provides an incremental improvement in understanding and bridging the performance gap between CSC models and CNNs for researchers and practitioners in image reconstruction.
This paper addresses the performance gap between Convolutional Sparse Coding (CSC) models and Convolutional Neural Networks (CNNs) in image reconstruction by introducing a multiscale convolutional dictionary structure derived from the U-Net. This new approach achieves performance competitive with state-of-the-art CNNs in challenging inverse problems like CT and MRI reconstruction.
Convolutional neural networks (CNNs) have been tremendously successful in solving imaging inverse problems. To understand their success, an effective strategy is to construct simpler and mathematically more tractable convolutional sparse coding (CSC) models that share essential ingredients with CNNs. Existing CSC methods, however, underperform leading CNNs in challenging inverse problems. We hypothesize that the performance gap may be attributed in part to how they process images at different spatial scales: While many CNNs use multiscale feature representations, existing CSC models mostly rely on single-scale dictionaries. To close the performance gap, we thus propose a multiscale convolutional dictionary structure. The proposed dictionary structure is derived from the U-Net, arguably the most versatile and widely used CNN for image-to-image learning problems. We show that incorporating the proposed multiscale dictionary in an otherwise standard CSC framework yields performance competitive with state-of-the-art CNNs across a range of challenging inverse problems including CT and MRI reconstruction. Our work thus demonstrates the effectiveness and scalability of the multiscale CSC approach in solving challenging inverse problems.